Fulcio CA certificate issue

[cooktheryan]

Currently, if a certificate is private or self signed fulcio will require the following. oc set env -n fulcio-system deployment/fulcio-server SSL_CERT_DIR=/var/run/fulcio

We need to try to see if the new Fulcio code fixes this or if we will need to code in our own fix

[osmman]

By using simple SSL_CERT_DIR can cause problem when Fulcio will use multiple OIDC which one is for example Google and internal Keycloak (custom CA root). When I will add custom CA root to SSL_CERT_DIR then it will override the default system's CA trusted roots so it will not trust Google OIDC and user will have to add another CA roots.

// certDirEnv is the environment variable which identifies which directory
// to check for SSL certificate files. If set this overrides the system default.
// It is a colon separated list of directories.
// See https://www.openssl.org/docs/man1.0.2/man1/c_rehash.html.
certDirEnv = "SSL_CERT_DIR"

https://github.com/golang/go/blob/master/src/crypto/x509/root_unix.go#L21-L25
https://github.com/golang/go/blob/master/src/crypto/x509/root_linux.go#L10-L23

[osmman]

Maybe the solution could be to mount user's CA file directly into /etc/pki/tls/certs directory. If I understand go's crypto code correctly than it should be loaded.

https://github.com/golang/go/blob/master/src/crypto/x509/root_linux.go#L22
https://github.com/golang/go/blob/master/src/crypto/x509/root_unix.go#L72

[osmman]

We cannot use /var/run/fulcio/ca.crt because Fulcio use these certificates only for https://kubernetes.default.svc OIDC (not generic).

https://github.com/sigstore/fulcio/blob/main/pkg/config/config.go#L226

[osmman]

I think I have found one solution to add extra trusted CA for the Fulcio server. The idea is to mount a new file with bundle of extra certificates into /etc/pki/tls/certs/ folder. The deployment configuration goes something like this:

spec:
  template:
    spec:
      containers:
      - name: fulcio-server
        volumeMounts:
        - name: trusted-ca
          mountPath: /etc/pki/tls/certs/fulcio-oidc.crt
          subPath: "fulcio-oidc.crt"
          readOnly: true
      volumes:
      - name: trusted-ca
        configMap:
          name: fulcio-trusted-ca
          items:
            - key: ca.crt
              path: "fulcio-oidc.crt"

Content of ca.crt

-----BEGIN CERTIFICATE-----
first cert
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
second cert
-----END CERTIFICATE-----

Now just figure out how to connect it to the Operator. I have in my mind these following possibilities:

1. simply method which expect that namespace has exactly prdefined config map. Similar solution which is used in oidc-info volume.
2. extend the Fulcio API by adding new field to add trusted CA.
   1. Example which enabled multiple config maps:

      kind: Fulcio
      spec:
        trustedCA:
          - name: kube-root-ca.crt
             key: ca.crt
          - name: openshift-service-ca.crt
             key: service-ca.crt
          - name: custom-ca-bundle
             key: ca.crt
      

   2. simplify version for only one CA bundle:

      kind: Fulcio
       spec:
         trustedCA:
           name: custom-ca-bundle
           key: ca.crt
      

@cooktheryan @bouskaJ @lance @Gregory-Pereira What do you think?

[osmman]

Now I noticed that it is also possible to use SSL_CERT_DIR env variable too because even if this value is set, the go crypto library will try to load these default os certificates.

This option could work to:

spec:
  template:
    spec:
      containers:
      - name: fulcio-server
        env:
          - name: SSL_CERT_DIR
             value: /var/run/fulcio
        volumeMounts:
        - name: trusted-ca
          mountPath: /var/run/fulcio/oidc.crt
          subPath: "oidc.crt"
          readOnly: true
      volumes:
      - name: trusted-ca
        configMap:
          name: fulcio-trusted-ca
          items:
            - key: ca.crt
              path: "oidc.crt"

[bouskaJ]

I would tend to use 2.ii - the bundle can be generated by https://cert-manager.io/docs/trust/trust-manager/ OR created manually (it does not matter).

Method 1 is not transparent.