From edb53a06d38f3886ed740051b5c169d289989dfc Mon Sep 17 00:00:00 2001 From: Paul Latzelsperger Date: Wed, 31 May 2023 08:31:22 +0200 Subject: [PATCH 1/4] docs: add multi-connector setup --- docs/samples/example-dataspace/README.md | 113 ++++++++++++ .../example-dataspace/daps/.helmignore | 23 +++ .../samples/example-dataspace/daps/Chart.yaml | 43 +++++ docs/samples/example-dataspace/daps/README.md | 39 ++++ .../daps/templates/_helpers.tpl | 62 +++++++ .../daps/templates/configmap.yaml | 92 ++++++++++ .../daps/templates/deployment.yaml | 168 ++++++++++++++++++ .../example-dataspace/daps/templates/hpa.yaml | 47 +++++ .../daps/templates/imagepullsecret.yaml | 31 ++++ .../daps/templates/service.yaml | 34 ++++ .../daps/templates/serviceaccount.yaml | 31 ++++ .../example-dataspace/daps/values.yaml | 96 ++++++++++ .../example-dataspace/plato-values.yaml | 78 ++++++++ .../example-dataspace/sokrates-values.yaml | 77 ++++++++ 14 files changed, 934 insertions(+) create mode 100644 docs/samples/example-dataspace/README.md create mode 100644 docs/samples/example-dataspace/daps/.helmignore create mode 100644 docs/samples/example-dataspace/daps/Chart.yaml create mode 100644 docs/samples/example-dataspace/daps/README.md create mode 100644 docs/samples/example-dataspace/daps/templates/_helpers.tpl create mode 100644 docs/samples/example-dataspace/daps/templates/configmap.yaml create mode 100644 docs/samples/example-dataspace/daps/templates/deployment.yaml create mode 100644 docs/samples/example-dataspace/daps/templates/hpa.yaml create mode 100644 docs/samples/example-dataspace/daps/templates/imagepullsecret.yaml create mode 100644 docs/samples/example-dataspace/daps/templates/service.yaml create mode 100644 docs/samples/example-dataspace/daps/templates/serviceaccount.yaml create mode 100644 docs/samples/example-dataspace/daps/values.yaml create mode 100644 docs/samples/example-dataspace/plato-values.yaml create mode 100644 docs/samples/example-dataspace/sokrates-values.yaml diff --git a/docs/samples/example-dataspace/README.md b/docs/samples/example-dataspace/README.md new file mode 100644 index 000000000..41c19b649 --- /dev/null +++ b/docs/samples/example-dataspace/README.md @@ -0,0 +1,113 @@ +## 1. Prepare environment: + +This guide will bring up two connectors named "Sokrates" and "Plato", each alongside their dependencies (Hashicorp +Vault, PostgreSQL) and a DAPS instance that both share. + +We've tested this setup with [KinD](https://kind.sigs.k8s.io/), but other runtimes such +as [Minikube](https://minikube.sigs.k8s.io/docs/start/) may work as well, but we haven't tested them. + +Furthermore, this guide assumes: + +- the Tractus-X EDC repository is checked out, the working directory for this guide is `docs/samples/example-dataspace` +- a Kubernetes runtime (e.g. KinD) is already installed and ready-to-use +- basic knowledge about `helm` and Kubernetes +- the following tools are available: `yq`, `openssl`, `base64` +- a POSIX-compliant shell, e.g. `bash` or `zsh` unless stated otherwise + +### 1.1 Create certificates for both runtimes + +We'll need a x509 certificate in order to communicate with DAPS, as well as a private key and a Data Encryption signing +key. + +```shell +# SOKRATES key/cert for daps +openssl req -newkey rsa:2048 -new -nodes -x509 -days 1 -keyout sokrates.key -out sokrates.cert -subj "/CN=test" +echo "aes_enckey_test" | base64 > sokrates.aes.key + +# PLATO key/cert for daps +openssl req -newkey rsa:2048 -new -nodes -x509 -days 1 -keyout plato.key -out plato.cert -subj "/CN=test" +echo "aes_enckey_test" | base64 > plato.aes.key +``` + +Any arbitrary string can be used for the AES key, but it has to be 16, 24, or 32 characters in length, assuming UTF-8 +encoding. + +### 1.2 Modify the DAPS's `values.yaml` located at `daps/values.yaml` + +With the following command, we "inject" the previously created certificates and client ids into the DAPS's config: + +```shell +VALUES_FILE=daps/values.yaml + +# Add both public keys to daps +yq -i ".connectors[0].certificate=\"$(cat sokrates.cert)\"" "$VALUES_FILE" +yq -i ".connectors[1].certificate=\"$(cat plato.cert)\"" "$VALUES_FILE" +``` + +### 1.3 Install/Launch DAPS + +`helm install daps daps/` + +## 2. Prepare Connectors + +Next, the certificates and private keys we created previously must be stored in each connector's vault by injecting +a `postStart` element to the chart's configuration file: + +```shell +# for sokrates +CONFIG_FILE=sokrates-values.yaml + +yq -i ".vault.server.postStart |= [\"sh\",\"-c\",\"{\nsleep 5\n\ncat << EOF | /bin/vault kv put secret/daps-crt content=-\n$(cat sokrates.cert)\nEOF\n\n +cat << EOF | /bin/vault kv put secret/daps-key content=-\n$(cat sokrates.key)\nEOF\n\n +/bin/vault kv put secret/aes-keys content=$(cat sokrates.aes.key)\n\n}\"]" "$CONFIG_FILE" + +# for plato +CONFIG_FILE=plato-values.yaml + +yq -i ".vault.server.postStart |= [\"sh\",\"-c\",\"{\nsleep 5\n\ncat << EOF | /bin/vault kv put secret/daps-crt content=-\n$(cat plato.cert)\nEOF\n\n +cat << EOF | /bin/vault kv put secret/daps-key content=-\n$(cat plato.key)\nEOF\n\n +/bin/vault kv put secret/aes-keys content=$(cat plato.aes.key)\n\n}\"]" "$CONFIG_FILE" +``` + +## 3 Install the connectors + +Use `helm` to install the Tractus-X EDC Helm charts. In this example we are using the _local_ charts, assuming you have +Tractus-X EDC checked out in your local filesystem at ``. + +```shell +# install sokrates +helm install tx-sokrates /charts/tractusx-connector \ + -f sokrates-values.yaml \ + --dependency-update + +# install plato +helm install tx-plato /charts/tractusx-connector \ + -f plato-values.yaml \ + --dependency-update +``` + +_Note: if you prefer to use the published version of the `tractusx-connector` chart, please add the Tractus-X Helm repo +first:_ + +```shell +helm repo add tractusx-edc https://eclipse-tractusx.github.io/charts/dev +helm install tx-[sokrates|plato] tractusx-edc/tractusx-connector \ + -f [sokrates|plato]-values.yaml \ + --dependency-update +``` + +## 3.1 [Optional] Verify the correct installation + +There is several ways of making sure everything worked out well: + +- simply look at the logs of the Helm releases, e.g. with a tool + like [stern](https://kubernetes.io/blog/2016/10/tail-kubernetes-with-stern/) and look out for a log line similar to: + ```shell + stern tx-sokates + ``` + then look out for something similar to: + ```shell + tx-sokrates-controlplane-b9456f97b-s5jts tractusx-connector INFO 2023-05-31T07:24:53.020975888 tx-sokrates-controlplane ready + ``` +- wait for the Kubernetes rollout to be successful, e.g. `kubectl rollout status deployment tx-plato-controlplane` +- use `helm test` to execute tests: `helm test tx-plato` \ No newline at end of file diff --git a/docs/samples/example-dataspace/daps/.helmignore b/docs/samples/example-dataspace/daps/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/docs/samples/example-dataspace/daps/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/docs/samples/example-dataspace/daps/Chart.yaml b/docs/samples/example-dataspace/daps/Chart.yaml new file mode 100644 index 000000000..f0a4e6e4e --- /dev/null +++ b/docs/samples/example-dataspace/daps/Chart.yaml @@ -0,0 +1,43 @@ +# Copyright (c) 2023 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +--- +apiVersion: v2 +name: daps +description: A Helm chart for Kubernetes + +# A chart can be either an 'application' or a 'library' chart. +# +# Application charts are a collection of templates that can be packaged into versioned archives +# to be deployed. +# +# Library charts provide useful utilities or functions for the chart developer. They're included as +# a dependency of application charts to inject those utilities and functions into the rendering +# pipeline. Library charts do not define any templates and therefore cannot be deployed. +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.0.1 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +# It is recommended to use it with quotes. +appVersion: "0.0.1" diff --git a/docs/samples/example-dataspace/daps/README.md b/docs/samples/example-dataspace/daps/README.md new file mode 100644 index 000000000..1f0c20e36 --- /dev/null +++ b/docs/samples/example-dataspace/daps/README.md @@ -0,0 +1,39 @@ +# daps + +![Version: 0.0.1](https://img.shields.io/badge/Version-0.0.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.0.1](https://img.shields.io/badge/AppVersion-0.0.1-informational?style=flat-square) + +A Helm chart for Kubernetes + +## Values + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| affinity | object | `{}` | [Affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) constrains which nodes the Pod can be scheduled on based on node labels. | +| automountServiceAccountToken | bool | `false` | Whether to [automount kubernetes API credentials](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server) into the pod | +| autoscaling.enabled | bool | `false` | Enables [horizontal pod autoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) | +| autoscaling.maxReplicas | int | `100` | Maximum replicas if resource consumption exceeds resource threshholds | +| autoscaling.minReplicas | int | `1` | Minimal replicas if resource consumption falls below resource threshholds | +| autoscaling.targetCPUUtilizationPercentage | int | `80` | targetAverageUtilization of cpu provided to a pod | +| autoscaling.targetMemoryUtilizationPercentage | int | `80` | targetAverageUtilization of memory provided to a pod | +| connectors | list | `[]` | | +| fullnameOverride | string | `""` | Overrides the releases full name | +| image.pullPolicy | string | `"IfNotPresent"` | [Kubernetes image pull policy](https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy) to use | +| image.repository | string | `"ghcr.io/fraunhofer-aisec/omejdn-server"` | Which omjedn container image to use | +| image.tag | string | `"1.7.1"` | Overrides the image tag whose default is the chart appVersion | +| imagePullSecret.dockerconfigjson | string | `""` | Image pull secret to create to [obtain the container image from private registries](https://kubernetes.io/docs/concepts/containers/images/#using-a-private-registry) Note: This value needs to adhere to the [(base64 encoded) .dockerconfigjson format](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#registry-secret-existing-credentials). Furthermore, if 'imagePullSecret.dockerconfigjson' is defined, it takes precedence over 'imagePullSecrets'. | +| nameOverride | string | `""` | Overrides the charts name | +| nodeSelector | object | `{}` | [Node-Selector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector) to constrain the Pod to nodes with specific labels. | +| podAnnotations | object | `{}` | [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) added to deployed [pods](https://kubernetes.io/docs/concepts/workloads/pods/) | +| podSecurityContext | object | `{}` | | +| replicaCount | int | `1` | Specifies how many replicas of a deployed pod shall be created during the deployment Note: If horizontal pod autoscaling is enabled this setting has no effect | +| resources | object | `{}` | [Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) applied to the deployed pod | +| securityContext | object | `{}` | | +| service.port | int | `4567` | [Service type](https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service) to expose the running application on a set of Pods as a network service. | +| service.type | string | `"ClusterIP"` | [Service type](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) to expose the running application on a set of Pods as a network service. | +| serviceAccount.annotations | object | `{}` | [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) to add to the service account | +| serviceAccount.create | bool | `true` | Specifies whether a [service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) should be created per release | +| serviceAccount.name | string | `""` | The name of the service account to use. If not set and create is true, a name is generated using the release's fullname template | +| tolerations | list | `[]` | [Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) are applied to Pods to schedule onto nodes with matching taints. | + +---------------------------------------------- +Autogenerated from chart metadata using [helm-docs v1.10.0](https://github.com/norwoodj/helm-docs/releases/v1.10.0) diff --git a/docs/samples/example-dataspace/daps/templates/_helpers.tpl b/docs/samples/example-dataspace/daps/templates/_helpers.tpl new file mode 100644 index 000000000..95b115eee --- /dev/null +++ b/docs/samples/example-dataspace/daps/templates/_helpers.tpl @@ -0,0 +1,62 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "omejdn.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "omejdn.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "omejdn.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels +*/}} +{{- define "omejdn.labels" -}} +helm.sh/chart: {{ include "omejdn.chart" . }} +{{ include "omejdn.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "omejdn.selectorLabels" -}} +app.kubernetes.io/name: {{ include "omejdn.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Create the name of the service account to use +*/}} +{{- define "omejdn.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "omejdn.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} diff --git a/docs/samples/example-dataspace/daps/templates/configmap.yaml b/docs/samples/example-dataspace/daps/templates/configmap.yaml new file mode 100644 index 000000000..5ad21648d --- /dev/null +++ b/docs/samples/example-dataspace/daps/templates/configmap.yaml @@ -0,0 +1,92 @@ +# Copyright (c) 2023 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "omejdn.fullname" . }} + labels: + {{- include "omejdn.labels" . | nindent 4 }} +data: + scope_mapping.yml: |- + --- + idsc:IDS_CONNECTOR_ATTRIBUTES_ALL: + - referringConnector + + omejdn.yml: |- + --- + host: http://daps:4567/ + path_prefix: '' + bind_to: 0.0.0.0 + allow_origin: "*" + app_env: debug + openid: false + user_backend: + - yaml + user_backend_default: yaml + accept_audience: idsc:IDS_CONNECTORS_ALL + issuer: http://daps:4567/ + environment: development + default_audience: + - idsc:IDS_CONNECTORS_ALL + access_token: + expiration: 3600 + algorithm: RS256 + id_token: + expiration: 3600 + algorithm: RS256 + + plugins.yml: |- + --- + plugins: + token_user_attributes: + + clients.yml: |- + --- + - client_id: data-plane-oauth2 + client_secret: supersecret + name: provision oauth2 + grant_types: + - client_credentials + token_endpoint_auth_method: client_secret_post + scope: openid +{{- range $i, $val := .Values.connectors }} + - client_id: {{ quote $val.id }} + name: {{ quote $val.name }} + token_endpoint_auth_method: private_key_jwt + grant_types: + - client_credentials + scope: + - idsc:IDS_CONNECTOR_ATTRIBUTES_ALL + attributes: + - key: idsc + value: IDS_CONNECTOR_ATTRIBUTES_ALL + - key: securityProfile + value: idsc:BASE_SECURITY_PROFILE + {{- range $key, $value := $val.attributes }} + - key: {{ $key }} + value: {{ $value }} + {{- end }} + redirect_uri: http://localhost:4200 +{{ end -}} + + +{{- range $i, $val := .Values.connectors }} + {{ $val.name }}: {{ quote $val.certificate | toString }} +{{ end -}} diff --git a/docs/samples/example-dataspace/daps/templates/deployment.yaml b/docs/samples/example-dataspace/daps/templates/deployment.yaml new file mode 100644 index 000000000..58bfff105 --- /dev/null +++ b/docs/samples/example-dataspace/daps/templates/deployment.yaml @@ -0,0 +1,168 @@ +# Copyright (c) 2023 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "omejdn.fullname" . }} + labels: + {{- include "omejdn.labels" . | nindent 4 }} +spec: + {{- if not .Values.autoscaling.enabled }} + replicas: {{ .Values.replicaCount }} + {{- end }} + selector: + matchLabels: + {{- include "omejdn.selectorLabels" . | nindent 6 }} + template: + metadata: + {{- with .Values.podAnnotations }} + annotations: + {{- toYaml . | nindent 8 }} + {{- end }} + labels: + {{- include "omejdn.selectorLabels" . | nindent 8 }} + spec: + {{- if .Values.imagePullSecret.dockerconfigjson }} + imagePullSecrets: + - name: {{ include "omejdn.fullname" . }}-imagepullsecret + {{- else }} + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- end }} + serviceAccountName: {{ include "omejdn.serviceAccountName" . }} + automountServiceAccountToken: {{ .Values.automountServiceAccountToken }} + securityContext: + {{- toYaml .Values.podSecurityContext | nindent 8 }} + {{- with .Values.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} + initContainers: + - name: init-daps-pvc + image: alpine + command: + - "sh" + - "-c" + args: + - | + cp /opt/config/omejdn.yml /etc/daps/omejdn.yml + cp /opt/config/clients.yml /etc/daps/clients.yml + cp /opt/config/plugins.yml /etc/daps/plugins.yml + cp /opt/config/scope_mapping.yml /etc/daps/scope_mapping.yml + apk add --update openssl + openssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -keyout /etc/keys/omejdn/omejdn.key \ + -subj "/C=DE/ST=Berlin/L=Berlin/O=Tractus-X-EDC-Test, Inc./OU=DE" + volumeMounts: + - mountPath: /etc/daps + name: config-dir + - mountPath: /etc/keys/omejdn + name: omejdn-key-dir + - mountPath: /opt/config/omejdn.yml + name: omejdn-config + subPath: omejdn.yml + - mountPath: /opt/config/scope_mapping.yml + name: scope-mapping + subPath: scope_mapping.yml + - mountPath: /opt/config/clients.yml + name: clients-config + subPath: clients.yml + - mountPath: /opt/config/plugins.yml + name: plugins-config + subPath: plugins.yml + containers: + - name: {{ .Chart.Name }} + securityContext: + {{- toYaml .Values.securityContext | nindent 12 }} + image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + volumeMounts: + - mountPath: /opt/config/ + name: config-dir + - mountPath: /opt/keys/omejdn/omejdn.key + name: omejdn-key-dir + subPath: omejdn.key + - mountPath: /opt/keys/clients/ + name: client-certificates + ports: + - name: http + containerPort: 4567 + protocol: TCP + livenessProbe: + httpGet: + path: /jwks.json + port: http + readinessProbe: + httpGet: + path: /jwks.json + port: http + resources: + {{- toYaml .Values.resources | nindent 12 }} + env: + - name: OMEJDN_JWT_AUD_OVERRIDE + value: "idsc:IDS_CONNECTORS_ALL" + - name: OMEJDN_PLUGINS + value: "config/plugins.yml" + volumes: + - name: config-dir + emptyDir: { } + - name: omejdn-key-dir + emptyDir: { } + - name: omejdn-config + configMap: + name: {{ include "omejdn.fullname" . }} + items: + - key: omejdn.yml + path: omejdn.yml + - name: scope-mapping + configMap: + name: {{ include "omejdn.fullname" . }} + items: + - key: scope_mapping.yml + path: scope_mapping.yml + - name: clients-config + configMap: + name: {{ include "omejdn.fullname" . }} + items: + - key: clients.yml + path: clients.yml + - name: plugins-config + configMap: + name: {{ include "omejdn.fullname" . }} + items: + - key: plugins.yml + path: plugins.yml + - name: client-certificates + configMap: + name: {{ include "omejdn.fullname" . }} + items: + {{- range $i, $val := .Values.connectors }} + - key: {{ $val.name }} + path: {{ $val.id }}.cert + {{- end }} diff --git a/docs/samples/example-dataspace/daps/templates/hpa.yaml b/docs/samples/example-dataspace/daps/templates/hpa.yaml new file mode 100644 index 000000000..f1f072f6c --- /dev/null +++ b/docs/samples/example-dataspace/daps/templates/hpa.yaml @@ -0,0 +1,47 @@ +# Copyright (c) 2023 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License, Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + # License for the specific language governing permissions and limitations + # under the License. + # + # SPDX-License-Identifier: Apache-2.0 + # + +{{- if .Values.autoscaling.enabled }} +--- +apiVersion: autoscaling/v2beta1 +kind: HorizontalPodAutoscaler +metadata: + name: {{ include "omejdn.fullname" . }} + labels: + {{- include "omejdn.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: {{ include "omejdn.fullname" . }} + minReplicas: {{ .Values.autoscaling.minReplicas }} + maxReplicas: {{ .Values.autoscaling.maxReplicas }} + metrics: + {{- if .Values.autoscaling.targetCPUUtilizationPercentage }} + - type: Resource + resource: + name: cpu + targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }} + {{- end }} + {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }} + - type: Resource + resource: + name: memory + targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }} + {{- end }} +{{- end }} diff --git a/docs/samples/example-dataspace/daps/templates/imagepullsecret.yaml b/docs/samples/example-dataspace/daps/templates/imagepullsecret.yaml new file mode 100644 index 000000000..44f573e0f --- /dev/null +++ b/docs/samples/example-dataspace/daps/templates/imagepullsecret.yaml @@ -0,0 +1,31 @@ +# Copyright (c) 2023 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License, Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + # License for the specific language governing permissions and limitations + # under the License. + # + # SPDX-License-Identifier: Apache-2.0 + # + +{{- if .Values.imagePullSecret.dockerconfigjson }} +--- +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "edc-dataplane.fullname" . }}-imagepullsecret + namespace: {{ .Release.Namespace | default "default" | quote }} + labels: + {{- include "edc-dataplane.labels" . | nindent 4 }} +data: + .dockerconfigjson: {{ .Values.imagePullSecret.dockerconfigjson }} +type: kubernetes.io/dockerconfigjson +{{- end }} diff --git a/docs/samples/example-dataspace/daps/templates/service.yaml b/docs/samples/example-dataspace/daps/templates/service.yaml new file mode 100644 index 000000000..947e69742 --- /dev/null +++ b/docs/samples/example-dataspace/daps/templates/service.yaml @@ -0,0 +1,34 @@ +# Copyright (c) 2023 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +--- +apiVersion: v1 +kind: Service +metadata: + name: {{ include "omejdn.fullname" . }} + labels: + {{- include "omejdn.labels" . | nindent 4 }} +spec: + type: {{ .Values.service.type }} + ports: + - port: {{ .Values.service.port }} + targetPort: http + protocol: TCP + name: http + selector: + {{- include "omejdn.selectorLabels" . | nindent 4 }} diff --git a/docs/samples/example-dataspace/daps/templates/serviceaccount.yaml b/docs/samples/example-dataspace/daps/templates/serviceaccount.yaml new file mode 100644 index 000000000..536f31871 --- /dev/null +++ b/docs/samples/example-dataspace/daps/templates/serviceaccount.yaml @@ -0,0 +1,31 @@ +# Copyright (c) 2023 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +{{- if .Values.serviceAccount.create -}} +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "omejdn.serviceAccountName" . }} + labels: + {{- include "omejdn.labels" . | nindent 4 }} + {{- with .Values.serviceAccount.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/docs/samples/example-dataspace/daps/values.yaml b/docs/samples/example-dataspace/daps/values.yaml new file mode 100644 index 000000000..3553dcc86 --- /dev/null +++ b/docs/samples/example-dataspace/daps/values.yaml @@ -0,0 +1,96 @@ +# Copyright (c) 2023 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +--- +# Default values for omejdn. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +# -- Specifies how many replicas of a deployed pod shall be created during the deployment +# Note: If horizontal pod autoscaling is enabled this setting has no effect +replicaCount: 1 +image: + # -- Which omjedn container image to use + repository: ghcr.io/fraunhofer-aisec/omejdn-server + # -- [Kubernetes image pull policy](https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy) to use + pullPolicy: IfNotPresent + # -- Overrides the image tag whose default is the chart appVersion + tag: "1.7.1" +imagePullSecret: + # -- Image pull secret to create to [obtain the container image from private registries](https://kubernetes.io/docs/concepts/containers/images/#using-a-private-registry) + # Note: This value needs to adhere to the [(base64 encoded) .dockerconfigjson format](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#registry-secret-existing-credentials). + # Furthermore, if 'imagePullSecret.dockerconfigjson' is defined, it takes precedence over 'imagePullSecrets'. + dockerconfigjson: "" +# -- Overrides the charts name +nameOverride: "" +# -- Overrides the releases full name +serviceAccount: + # -- Specifies whether a [service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) should be created per release + create: true + # -- [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) to add to the service account + annotations: { } + # -- The name of the service account to use. If not set and create is true, a name is generated using the release's fullname template + name: "" +# -- Whether to [automount kubernetes API credentials](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server) into the pod +automountServiceAccountToken: false +# -- [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) added to deployed [pods](https://kubernetes.io/docs/concepts/workloads/pods/) +podAnnotations: { } +# The [pod security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) defines privilege and access control settings for a Pod within the deployment +podSecurityContext: { } +# The [container security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) defines privilege and access control settings for a Container within a pod +securityContext: { } +service: + # -- [Service type](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) to expose the running application on a set of Pods as a network service. + type: ClusterIP + # -- [Service type](https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service) to expose the running application on a set of Pods as a network service. + port: 4567 +# -- [Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) applied to the deployed pod +resources: { } +autoscaling: + # -- Enables [horizontal pod autoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) + enabled: false + # -- Minimal replicas if resource consumption falls below resource threshholds + minReplicas: 1 + # -- Maximum replicas if resource consumption exceeds resource threshholds + maxReplicas: 100 + # -- targetAverageUtilization of cpu provided to a pod + targetCPUUtilizationPercentage: 80 + # -- targetAverageUtilization of memory provided to a pod + targetMemoryUtilizationPercentage: 80 +# -- [Node-Selector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector) to constrain the Pod to nodes with specific labels. +nodeSelector: { } +# -- [Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) are applied to Pods to schedule onto nodes with matching taints. +tolerations: [ ] +# -- [Affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) constrains which nodes the Pod can be scheduled on based on node labels. +affinity: { } +# List of connector clients. Certificate and Client-ID must be configured in parallel. +fullnameOverride: "daps" +url: "" +clientId: "" +paths: + jwks: /jwks.json + token: /token +connectors: + - id: E7:07:2D:74:56:66:31:F0:7B:10:EA:B6:03:06:4C:23:7F:ED:A6:65:keyid:E7:07:2D:74:56:66:31:F0:7B:10:EA:B6:03:06:4C:23:7F:ED:A6:65 + name: sokrates + attributes: + referringConnector: http://sokrates-controlplane/BPNSOKRATES + - id: E7:07:2D:74:56:66:31:F0:7B:10:EA:B6:03:06:4C:23:7F:ED:A6:65:keyid:E7:07:2D:74:56:66:31:F0:7B:10:EA:B6:03:06:4C:23:7F:ED:A6:69 + name: plato + attributes: + referringConnector: http://plato-controlplane/BPNPLATO diff --git a/docs/samples/example-dataspace/plato-values.yaml b/docs/samples/example-dataspace/plato-values.yaml new file mode 100644 index 000000000..ce2838962 --- /dev/null +++ b/docs/samples/example-dataspace/plato-values.yaml @@ -0,0 +1,78 @@ +# +# Copyright (c) 2023 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + + +install: + daps: false +fullnameOverride: tx-plato +################################ +# EDC ControlPlane + DataPlane # +################################ +participant: + id: "test-participant" +controlplane: + service: + type: NodePort + endpoints: + management: + authKey: password + image: + pullPolicy: Never + tag: "latest" + repository: "edc-controlplane-postgresql-hashicorp-vault" + securityContext: + # avoids some errors in the log: cannot write temp files of large multipart requests when R/O + readOnlyRootFilesystem: false +dataplane: + image: + pullPolicy: Never + tag: "latest" + repository: "edc-dataplane-hashicorp-vault" + securityContext: + # avoids some errors in the log: cannot write temp files of large multipart requests when R/O + readOnlyRootFilesystem: false + aws: + endpointOverride: http://minio:9000 + secretAccessKey: qwerty123 + accessKeyId: qwerty123 +postgresql: + fullnameOverride: "plato-postgresql" + username: user + password: password + jdbcUrl: jdbc:postgresql://plato-postgresql:5432/edc +vault: + fullnameOverride: "plato-vault" + hashicorp: + url: http://plato-vault:8200 + token: root + secretNames: + transferProxyTokenSignerPublicKey: daps-crt + transferProxyTokenSignerPrivateKey: daps-key + transferProxyTokenEncryptionAesKey: aes-keys + dapsPrivateKey: daps-key + dapsPublicKey: daps-crt + # this must be set through CLI args: --set vault.secrets=$YOUR_VAULT_SECRETS where YOUR_VAULT_SECRETS should + # be a string in the format "key1:secret1;key2:secret2;..." + secrets: + server: +daps: + url: "http://daps:4567" + clientId: "E7:07:2D:74:56:66:31:F0:7B:10:EA:B6:03:06:4C:23:7F:ED:A6:65:keyid:E7:07:2D:74:56:66:31:F0:7B:10:EA:B6:03:06:4C:23:7F:ED:A6:69" +backendService: + httpProxyTokenReceiverUrl: "http://backend:8080" diff --git a/docs/samples/example-dataspace/sokrates-values.yaml b/docs/samples/example-dataspace/sokrates-values.yaml new file mode 100644 index 000000000..7d9b1427d --- /dev/null +++ b/docs/samples/example-dataspace/sokrates-values.yaml @@ -0,0 +1,77 @@ +# +# Copyright (c) 2023 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +install: + daps: false +fullnameOverride: tx-sokrates +################################ +# EDC ControlPlane + DataPlane # +################################ +participant: + id: "test-participant" +controlplane: + service: + type: NodePort + endpoints: + management: + authKey: password + image: + pullPolicy: Never + tag: "latest" + repository: "edc-controlplane-postgresql-hashicorp-vault" + securityContext: + # avoids some errors in the log: cannot write temp files of large multipart requests when R/O + readOnlyRootFilesystem: false +dataplane: + image: + pullPolicy: Never + tag: "latest" + repository: "edc-dataplane-hashicorp-vault" + securityContext: + # avoids some errors in the log: cannot write temp files of large multipart requests when R/O + readOnlyRootFilesystem: false + aws: + endpointOverride: http://minio:9000 + secretAccessKey: qwerty123 + accessKeyId: qwerty123 +postgresql: + fullnameOverride: "sokrates-postgresql" + username: user + password: password + jdbcUrl: jdbc:postgresql://sokrates-postgresql:5432/edc +vault: + fullnameOverride: "sokrates-vault" + hashicorp: + url: http://sokrates-vault:8200 + token: root + secretNames: + transferProxyTokenSignerPublicKey: daps-crt + transferProxyTokenSignerPrivateKey: daps-key + transferProxyTokenEncryptionAesKey: aes-keys + dapsPrivateKey: daps-key + dapsPublicKey: daps-crt + # this must be set through CLI args: --set vault.secrets=$YOUR_VAULT_SECRETS where YOUR_VAULT_SECRETS should + # be a string in the format "key1:secret1;key2:secret2;..." + secrets: + server: +daps: + url: "http://daps:4567" + clientId: "E7:07:2D:74:56:66:31:F0:7B:10:EA:B6:03:06:4C:23:7F:ED:A6:65:keyid:E7:07:2D:74:56:66:31:F0:7B:10:EA:B6:03:06:4C:23:7F:ED:A6:65" +backendService: + httpProxyTokenReceiverUrl: "http://backend:8080" From 1562c5b88409415704a42b9de8eb2b1d325cc468 Mon Sep 17 00:00:00 2001 From: Paul Latzelsperger Date: Wed, 31 May 2023 11:23:57 +0200 Subject: [PATCH 2/4] markdown lint --- docs/samples/example-dataspace/README.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/samples/example-dataspace/README.md b/docs/samples/example-dataspace/README.md index 41c19b649..f61badb33 100644 --- a/docs/samples/example-dataspace/README.md +++ b/docs/samples/example-dataspace/README.md @@ -1,4 +1,6 @@ -## 1. Prepare environment: +# How-To run two connectors and a DAPS + +## 1. Prepare environment This guide will bring up two connectors named "Sokrates" and "Plato", each alongside their dependencies (Hashicorp Vault, PostgreSQL) and a DAPS instance that both share. @@ -102,12 +104,16 @@ There is several ways of making sure everything worked out well: - simply look at the logs of the Helm releases, e.g. with a tool like [stern](https://kubernetes.io/blog/2016/10/tail-kubernetes-with-stern/) and look out for a log line similar to: - ```shell - stern tx-sokates + + ```shell + stern tx-sokrates ``` + then look out for something similar to: + ```shell tx-sokrates-controlplane-b9456f97b-s5jts tractusx-connector INFO 2023-05-31T07:24:53.020975888 tx-sokrates-controlplane ready ``` + - wait for the Kubernetes rollout to be successful, e.g. `kubectl rollout status deployment tx-plato-controlplane` -- use `helm test` to execute tests: `helm test tx-plato` \ No newline at end of file +- use `helm test` to execute tests: `helm test tx-plato` From 4b03eec8ea82cd36b0df6ec561370d9e0784b6fa Mon Sep 17 00:00:00 2001 From: Paul Latzelsperger Date: Wed, 31 May 2023 11:28:24 +0200 Subject: [PATCH 3/4] textual improvement --- docs/samples/example-dataspace/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/samples/example-dataspace/README.md b/docs/samples/example-dataspace/README.md index f61badb33..9842186cd 100644 --- a/docs/samples/example-dataspace/README.md +++ b/docs/samples/example-dataspace/README.md @@ -6,7 +6,7 @@ This guide will bring up two connectors named "Sokrates" and "Plato", each along Vault, PostgreSQL) and a DAPS instance that both share. We've tested this setup with [KinD](https://kind.sigs.k8s.io/), but other runtimes such -as [Minikube](https://minikube.sigs.k8s.io/docs/start/) may work as well, but we haven't tested them. +as [Minikube](https://minikube.sigs.k8s.io/docs/start/) may work as well, we just haven't tested them. Furthermore, this guide assumes: From 13150958e26c2b56f60d44a96e4154876e17fa62 Mon Sep 17 00:00:00 2001 From: Paul Latzelsperger <43503240+paullatzelsperger@users.noreply.github.com> Date: Wed, 31 May 2023 12:03:11 +0200 Subject: [PATCH 4/4] Apply suggestions from code review Co-authored-by: Enrico Risa --- docs/samples/example-dataspace/plato-values.yaml | 2 +- docs/samples/example-dataspace/sokrates-values.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/samples/example-dataspace/plato-values.yaml b/docs/samples/example-dataspace/plato-values.yaml index ce2838962..21c5675d7 100644 --- a/docs/samples/example-dataspace/plato-values.yaml +++ b/docs/samples/example-dataspace/plato-values.yaml @@ -25,7 +25,7 @@ fullnameOverride: tx-plato # EDC ControlPlane + DataPlane # ################################ participant: - id: "test-participant" + id: "BPNPLATO" controlplane: service: type: NodePort diff --git a/docs/samples/example-dataspace/sokrates-values.yaml b/docs/samples/example-dataspace/sokrates-values.yaml index 7d9b1427d..086eefde5 100644 --- a/docs/samples/example-dataspace/sokrates-values.yaml +++ b/docs/samples/example-dataspace/sokrates-values.yaml @@ -24,7 +24,7 @@ fullnameOverride: tx-sokrates # EDC ControlPlane + DataPlane # ################################ participant: - id: "test-participant" + id: "BPNSOKRATES" controlplane: service: type: NodePort