Custom SSL Root CA Certificates overrule certificates from default cacerts when running in native mode

[chberger]

Describe the bug

According to this guide it should be possible to add a custom Root CA certificate to a Quarkus native application. Furthermore its also mentioned when passing the certificates via the following system properties to the underlying GraalVM:
quarkus.native.additional-build-args=-J-Djavax.net.ssl.trustStore=/tmp/mycerts,-J-Djavax.net.ssl.trustStorePassword=changeit
the certificates of /tmp/mycerts are baked into the native binary and used in addition to the default cacerts.

However, that's not what I've observed. When passing a custom trust-store to the native build only those certificates which are part of the custom trust-store are present at runtime. All other trusted ca-certs, which are available during the build (e.g. provided by GrallVM) are gone, or at least not accessible by the resulting native app.

Expected behavior

According to the documentation my expectation would be that certificates from a custom trust-store are being used in addition to any other trusted ca-certficate, that has been provided by the default cacerts during the native image build.

For example: Let's imagine that the GraalVM ships with a trust-store which contains 129 Root CA certificates. Since I pass a custom trust-store with a single certificate my expectation would be that all those 130 Root CA certificates are baked in the native binary.

Actual behavior

The certificates of a custom trust-store overrule cacerts present in the GraalVM. The ca-certificates aren't baked into the native binary (or at least they aren't recognized at runtime) as they should according to the documentation. Just the custom certificates are present..

How to Reproduce?

1. Add required configs in application.properties (I'm using jib to create the image, maybe that's of relevance)

quarkus.container-image.build=true
quarkus.container-image.push=false
quarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-native-image:22.3-java17
quarkus.native.additional-build-args=-J-Djavax.net.ssl.trustStore=src/main/resources/META-INF/truststore.jks,-J-Djavax.net.ssl.trustStorePassword=changeit
quarkus.jib.base-native-image=quay.io/quarkus/quarkus-distroless-image:2.0
quarkus.jib.base-jvm-image=**Any JVM base image should work**

2. Add a class which prints the certificates available at build and runtime:
   I had no better idea to extract the certificates from the native binary. From my understanding they are baked into the binary and not being loaded from the underlying OS default trust-store.

@ApplicationScoped
public class TLSStartup {

    void onStart(@Observes StartupEvent ev) throws NoSuchAlgorithmException, KeyStoreException {
        TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        trustManagerFactory.init((KeyStore) null);

        List<TrustManager> trustManagers = Arrays.asList(trustManagerFactory.getTrustManagers());
        List<X509Certificate> certificates = trustManagers.stream()
                .filter(X509TrustManager.class::isInstance)
                .map(X509TrustManager.class::cast)
                .map(trustManager -> Arrays.asList(trustManager.getAcceptedIssuers()))
                .flatMap(Collection::stream)
                .collect(Collectors.toList());
        System.out.println("certificates -> "+certificates);
    }
}

3. Trigger a native build. I'm using the container approach since I haven't installed GraalVM locally:
   mvn package -Pnative -Dquarkus.native.container-build=true

4. The defined observer shall be trigger at build and runtime. During the native image build check the sys out log and verify that the expected ca-certs are present. Once the binary has been crafted start the image and check the output. In that case only the certs from the custom trust-store should be present.

Output of uname -a or ver

No response

Output of java -version

17.0.5+8-jvmci-22.3-b08

GraalVM version (if different from Java)

GraalVM 22.3.0 Java 17 CE

Quarkus version or git rev

2.13.3.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.8.5 (3599d3414f046de2324203b78ddcf9b5e4388aa0)

Additional information

Docker Desktop on Windows 10:
Server Version: 20.10.20
API Version: 1.41
Operating System: Docker Desktop

[quarkus-bot]

/cc @geoand

[geoand]

cc @galderz @zakkak

[zakkak]

To my understanding this is wrongfully stated in the Quarkus documentation.

Quoting GraalVM's documentation:

During the image building process, the native-image builder captures the host environment’s default TrustStore and embeds it into the native executable. This TrustStore is by default created from the root certificate file provided within the JDK, but can be changed to use a different certificate file by setting the build-time system property javax.net.ssl.trustStore (see Properties for how to do it).

[chberger]

Fair enough if it works as designed. I'm just wondering why it is stated so explicitly in the Quarkus docs ...

[zakkak]

It looks like this was added in aa5820e#diff-da5c166ecda36541821eb24af6ff0c775f7759d4a95edda2e3f7b7cc1bc6c310R183 by @geoand and @gsmet, Georgios and Guillaume does any of you recall what lead to this statement?

Shall we just change the guide to reflect the GraalVM docs?

[geoand]

Yeah, I think that makes sense.