Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(helm): rework chart to use latest tractusx-connector chart #22

Merged
merged 11 commits into from
Jun 26, 2023

Conversation

fty4
Copy link
Member

@fty4 fty4 commented Jun 14, 2023

TL;DR

The current e2e-testing is using separate daps, psql and vault dependencies - these are now provided inside the tractusx-connector chart.
With this PR the dependencies of the tx-c chart will be used instead of creating own dependencies.

Description

Currently the Helm chart is using its own daps-server, postgresql and vault dependency.
That was the case because the edc chart did not yet provide those.
But that changed since v0.4.1 was released.

Therefore instead of providing separate dependencies to the tractusx-connector chart its dependencies can be used for the e2e-testing umbrella chart.

Due the fact that only one daps-server and vault is required both are deactivated on the edcprovider dependency.
Each edc will receive its own psql.

Currently there are some changes which are required to be able to deploy this new structure.
daps, postgresql and vault need to allow templating in their application fullname.
This also will allow all components share the same prefix ({{ .Release.Name }}-).
Also the configuration in the values file will be simplified.
The PRs are linked below.

Test PR

Currently there is a way to test the new umbrella chart with a test-release from mercedes-benz/eclipse-tractusx_tractusx-edc-merge:

Test instructions

Chart.yaml

Update your Chart.yaml of the umbrella chart as follows:

dependencies:
  # ...

  # edc consumer
  - alias: edcconsumer
    name: tractusx-connector
-   version: 0.4.1
+   version: 0.4.1-PR473-474-475
-   repository: https://eclipse-tractusx.github.io/charts/dev
+   repository: https://mercedes-benz.github.io/eclipse-tractusx_tractusx-edc-merge
  # edc provider
  - alias: edcprovider
    name: tractusx-connector
-   version: 0.4.1
+   version: 0.4.1-PR473-474-475
-   repository: https://eclipse-tractusx.github.io/charts/dev
+   repository: https://mercedes-benz.github.io/eclipse-tractusx_tractusx-edc-merge

Note: Version 0.4.1-PR473-474-475 represents v0.4.1 with all of the required PRs merged.

values-patch-image.yaml

Because the version is referencing its own version also the values file requires following additional configuration (e.g. additional value.yaml-file):

global:
  tag: &imageTag 0.4.1

edcconsumer:
  controlplane:
    image:
      tag: *imageTag

  dataplane:
    image:
      tag: *imageTag

edcprovider:
  controlplane:
    image:
      tag: *imageTag

  dataplane:
    image:
      tag: *imageTag

Installation

$ helm dependency update ./charts/umbrella/

$ helm install e2etesting ./charts/umbrella \
      --values ./charts/umbrella/values.yaml \
      --values ./charts/umbrella/values.patch-image.yaml

Dependencies

As mentioned following changes in the tractusx-connector Helm chart are required to be released:


Marco Lecheler [email protected] Mercedes-Benz Tech Innovation GmbH (ProviderInformation)

Copy link
Contributor

@mbctec mbctec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should rename the chart to someting meaningfull in the future

@fty4
Copy link
Member Author

fty4 commented Jun 15, 2023

Maybe we should rename the chart to someting meaningfull in the future

see issue #23

@fty4
Copy link
Member Author

fty4 commented Jun 23, 2023

edc 0.4.2 was not yet released but 0.5.0-rc1 was.

This version does not use daps-server anymore but Managed Identity Walled (MIW) / SSI.
Also there is a tractusx-connector-legacy chart which might be used for this PR until the MIW was integrated or 0.4.2 is released.

@fty4
Copy link
Member Author

fty4 commented Jun 23, 2023

I just tried to use Helm chart 0.5.0-rc1 with images from 0.4.1.

But the dataplane will not start.
Following message will appear in the log:

No setting found for key edc.datasource.asset.name

full logs
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended                                                                                                                 
[otel.javaagent 2023-06-23 14:37:38:021 +0000] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 1.12.1                                                                                           
WARNING 2023-06-23T14:37:38.178262418 Configuration file does not exist: /app/configuration.properties. Ignoring.                                                                                                                                 
INFO 2023-06-23T14:37:38.285525543 Initialized FS Configuration                                                                                                                                                                                   
WARNING 2023-06-23T14:37:38.429040709 The runtime is configured as an anonymous participant. DO NOT DO THIS IN PRODUCTION.                                                                                                                        
INFO 2023-06-23T14:37:39.829011043 Initialized Micrometer Metrics                                                                                                                                                                                 
INFO 2023-06-23T14:37:39.83435446 Initialized Boot Services                                                                                                                                                                                       
INFO 2023-06-23T14:37:41.674749128 HashicorpVaultExtension: authentication/initialization complete.                                                                                                                                               
INFO 2023-06-23T14:37:41.678097336 Initialized Hashicorp Vault                                                                                                                                                                                    
INFO 2023-06-23T14:37:41.680033086 Initialized Core Default Services                                                                                                                                                                              
INFO 2023-06-23T14:37:41.697733794 HTTPS enforcement it not enabled, please enable it in a production environment                                                                                                                                 
INFO 2023-06-23T14:37:41.737239336 HTTPS enforcement it not enabled, please enable it in a production environment                                                                                                                                 
INFO 2023-06-23T14:37:41.798611961 Initialized Core Services                                                                                                                                                                                      
WARNING 2023-06-23T14:37:41.806906961 Settings: No setting found for key 'edc.hostname'. Using default value 'localhost'                                                                                                                          
INFO 2023-06-23T14:37:41.849828169 Initialized JSON-LD Extension                                                                                                                                                                                  
WARNING 2023-06-23T14:37:42.115725878 Failed to register cached json-ld document: Cannot read resource document/odrl.jsonld:                                                                                                                      
INFO 2023-06-23T14:37:42.122237169 Initialized Local Transaction                                                                                                                                                                                  
INFO 2023-06-23T14:37:42.123298253 Initialized SQL Core                                                                                                                                                                                           
INFO 2023-06-23T14:37:42.141158961 Initialized org.eclipse.edc.connector.store.sql.contractnegotiation.SqlContractNegotiationStoreExtension                                                                                                       
INFO 2023-06-23T14:37:42.141941128 Initialized Control Plane Default Services                                                                                                                                                                     
INFO 2023-06-23T14:37:42.146043419 Initialized org.eclipse.edc.connector.contract.ContractNegotiationCommandExtension                                                                                                                             
SEVERE 2023-06-23T14:37:42.153257128 Error booting runtime: No setting found for key edc.datasource.asset.name                                                                                                                                    
org.eclipse.edc.spi.EdcException: No setting found for key edc.datasource.asset.name                                                                                                                                                              
    at org.eclipse.edc.spi.system.configuration.ConfigImpl.getNotNullValue(ConfigImpl.java:198)                                                                                                                                                   
    at org.eclipse.edc.spi.system.configuration.ConfigImpl.getString(ConfigImpl.java:58)                                                                                                                                                          
    at org.eclipse.edc.connector.store.sql.assetindex.SqlAssetIndexServiceExtension.initialize(SqlAssetIndexServiceExtension.java:53)                                                                                                             
    at org.eclipse.edc.boot.system.injection.lifecycle.InitializePhase.initialize(InitializePhase.java:37)                                                                                                                                        
    at org.eclipse.edc.boot.system.injection.lifecycle.ExtensionLifecycleManager.initialize(ExtensionLifecycleManager.java:61)                                                                                                                    
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)                                                                                                                                                                    
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)                                                                                                                                                                    
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)                                                                                                                                                                    
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)                                                                                                                                                        
    at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)                                                                                                                                                                       
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)                                                                                                                                                                
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)                                                                                                                                                           
    at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)                                                                                                                                                                       
    at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)                                                                                                                                                                       
    at org.eclipse.edc.boot.system.ExtensionLoader.bootServiceExtensions(ExtensionLoader.java:67)                                                                                                                                                 
    at org.eclipse.edc.boot.system.runtime.BaseRuntime.bootExtensions(BaseRuntime.java:141)                                                                                                                                                       
    at org.eclipse.edc.boot.system.runtime.BaseRuntime.boot(BaseRuntime.java:202)                                                                                                                                                                 
    at org.eclipse.edc.boot.system.runtime.BaseRuntime.boot(BaseRuntime.java:84)                                                                                                                                                                  
    at org.eclipse.edc.boot.system.runtime.BaseRuntime.main(BaseRuntime.java:72)                                                                                                                                                                  
Stream closed EOF for default/laab-edcconsumer-dataplane-85cd68678c-g7wgb (edcconsumer)

@SebastianBezold
Copy link
Contributor

Hi @fty4,

I think removing our custom sub Charts for products, that already provide their own Chart is great! Using the existing ones that are maintained by the product teams as dependency should be our default.
I would be in favor of merging this PR. Some more thoughts below.

Regarding the EDC version:
There is still a lot of work going on to fully enable SSI as I heard. So there are two options. 1) stick with an older DAPS dependent version or 2) Already integrate the latest EDC version prepared for SSI.

  1. would allow us to start doing the first steps for actual e2e test automation
  2. would give us a head start for our next release, when SSI has to be e2e tested. The Chart would partly be available already

Since there is not really any activity (that I know of) in regards to test automation, I guess the better option to go for is 2). If at least the SSI enabled EDC is starting up and is integrated as a dependency, that would be fine in my opinion.

Regarding external transitive dependencies:
I think in the beginning it is totally fine, if product Charts bring their own transitive dependencies like Vault or PostgreSQL. This would make the setup easy. However, there might be dependencies, that have to exist as a single instance, like Keycloak for example. In that case, providing it via umbrella Chart and configuring the products to use this instance instead of their own is a must.
One of our long term goals is also to simulate a potential production hosting environment, where you could have a single centrally managed PostgreSQL instance, that contains multiple schemas for each product. So somewhere in the future, the e2e Chart should also enable a single PosgreSQL dependency instead of the products bringing their own. But as said, this is a long term goal and should be handled in a separate issue in my opinion.

FYI @Siegfriedk

temporary using fixed image + tag
Because there is no tractusx-connector release including PRs 473,474,475
for 0.4.x we use the legacy Helm chart from release 0.5.0-rc1
@fty4 fty4 marked this pull request as ready for review June 26, 2023 09:05
@fty4
Copy link
Member Author

fty4 commented Jun 26, 2023

I've just updated the PR to use the 0.5.0-rc1 Helm chart which includes the mentioned PRs 473,474,475 from the EDC repo.
In commit 0ca5ca2 the image will be overwritten to use 0.4.1 again.

The PR is now not longer a draft and can be reviewed and merged.

@SebastianBezold I think we should first merge this PR then the daps still will be in place.
When the edc is released with 0.5.0 we can aim in a separate PR to update the edc to use SSI.
I also agree to get a head start with SSI than using daps any longer.

@SebastianBezold
Copy link
Contributor

I've just updated the PR to use the 0.5.0-rc1 Helm chart which includes the mentioned PRs 473,474,475 from the EDC repo. In commit 0ca5ca2 the image will be overwritten to use 0.4.1 again.

The PR is now not longer a draft and can be reviewed and merged.

@SebastianBezold I think we should first merge this PR then the daps still will be in place. When the edc is released with 0.5.0 we can aim in a separate PR to update the edc to use SSI. I also agree to get a head start with SSI than using daps any longer.

Hi @fty4, agree. Could you please create issues to track the "new Chart version but 'old' EDC version"?

@SebastianBezold SebastianBezold merged commit c5a5d06 into eclipse-tractusx:main Jun 26, 2023
@fty4 fty4 deleted the feat/rework branch June 26, 2023 12:18
@fty4
Copy link
Member Author

fty4 commented Jun 26, 2023

issue created: #27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants