Kong OIDC cluster plugin does not appear to allow self signed certificates

[heretic098]

Hello. I am trying to set up a local development system including kong (installed via helm chart and configured through k8s resources) and keycloak as an openid connect (OIDC) identity provider (IDP).

Here is my kong cluster plugin configuration with sensitive parts redacted:

apiVersion: configuration.konghq.com/v1
config:
  bearer_jwt_auth_allowed_auds:
  - account
  bearer_jwt_auth_enable: "yes"
  client_id: kong
  client_secret: <client secret>
  discovery: https://auth.localdev/realms/myrealm/.well-known/openid-configuration
  realm: myrealm
  redirect_after_logout_uri: https://auth.localdev/realms/myrealm/protocol/openid-connect/token/introspect
  scope: openid
  ssl_verify: "False"
disabled: false
kind: KongClusterPlugin
metadata:
  annotations:
    kubernetes.io/ingress.class: kong
  labels:
    global: "false"
  name: mykcp
plugin: oidc

Here are the log lines I get from the proxy container:

023/03/09 10:57:30 [debug] 1098#0: *37509 [lua] base_plugin.lua:26: access(): executing plugin "oidc": access
2023/03/09 10:57:30 [debug] 1098#0: *37509 [lua] handler.lua:94: make_oidc(): OidcHandler calling authenticate, requested path: /
2023/03/09 10:57:30 [debug] 1098#0: *37509 [lua] openidc.lua:1511: authenticate(): session.present=nil, session.data.id_token=false, session.data.authenticated=nil, opts.force_reauthorize=nil, opts.renew_access_token_on_expiry=nil, try_to_renew=true, token_expired=false
2023/03/09 10:57:30 [debug] 1098#0: *37509 [lua] openidc.lua:560: openidc_discover(): openidc_discover: URL is: https://auth.localdev/realms/myrealm/.well-known/openid-configuration
2023/03/09 10:57:30 [debug] 1098#0: *37509 [lua] openidc.lua:566: openidc_discover(): discovery data not in cache, making call to discovery endpoint
2023/03/09 10:57:30 [debug] 1098#0: *37509 [lua] openidc.lua:434: openidc_configure_proxy(): openidc_configure_proxy : don't use http proxy
2023/03/09 10:57:36 [error] 1098#0: *37509 [lua] openidc.lua:577: openidc_discover(): accessing discovery url (https://auth.localdev/realms/myrealm/.well-known/openid-configuration) failed: 18: self signed certificate, client: 10.42.0.2, server: kong, request: "GET / HTTP/2.0", host: "myrealm.localdev"
2023/03/09 10:57:36 [info] 1097#0: *37582 client closed connection while waiting for request, client: 10.42.0.1, server: 0.0.0.0:8443
10.42.0.2 - - [09/Mar/2023:10:57:36 +0000] "GET / HTTP/2.0" 500 191 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0"
2023/03/09 10:57:36 [debug] 1098#0: *37509 [lua] base_plugin.lua:26: access(): executing plugin "oidc": access
2023/03/09 10:57:36 [debug] 1098#0: *37509 [lua] handler.lua:94: make_oidc(): OidcHandler calling authenticate, requested path: /favicon.ico
2023/03/09 10:57:36 [debug] 1098#0: *37509 [lua] openidc.lua:1511: authenticate(): session.present=true, session.data.id_token=false, session.data.authenticated=nil, opts.force_reauthorize=nil, opts.renew_access_token_on_expiry=nil, try_to_renew=true, token_expired=false
2023/03/09 10:57:36 [debug] 1098#0: *37509 [lua] openidc.lua:560: openidc_discover(): openidc_discover: URL is: https://auth.localdev/realms/myrealm/.well-known/openid-configuration
2023/03/09 10:57:36 [debug] 1098#0: *37509 [lua] openidc.lua:566: openidc_discover(): discovery data not in cache, making call to discovery endpoint
2023/03/09 10:57:36 [debug] 1098#0: *37509 [lua] openidc.lua:434: openidc_configure_proxy(): openidc_configure_proxy : don't use http proxy
2023/03/09 10:57:36 [error] 1098#0: *37509 [lua] openidc.lua:577: openidc_discover(): accessing discovery url (https://auth.localdev/realms/myrealm/.well-known/openid-configuration) failed: 18: self signed certificate, client: 10.42.0.2, server: kong, request: "GET /favicon.ico HTTP/2.0", host: "myrealm.localdev", referrer: "https://myrealm.localdev/"
2023/03/09 10:57:36 [info] 1097#0: *37585 client closed connection while waiting for request, client: 10.42.0.1, server: 0.0.0.0:8443
10.42.0.2 - - [09/Mar/2023:10:57:36 +0000] "GET /favicon.ico HTTP/2.0" 500 46 "https://myrealm.localdev/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0"

Particularly, it says accessing discovery url ... failed: 18: self signed certificate despite the config.ssl_verify property on the KCP having been set to false per the docs.

As this is a local development environment, keycloak is using a self signed certificate and it is allowable (in my opinion) in this instance. Is there a reason that the config.ssl_verify property does not seem to be being honoured for the OIDC kong cluster plugin? Is there something else I need to do in order to make this work please?

[ghost]

try to add you custom CA certs to lua_ssl_trusted_certificate, PTAL: https://docs.konghq.com/gateway/latest/reference/configuration/#lua_ssl_trusted_certificate

[ghost]

and your config.ssl_verify: "False"? I think we should use false here?

[heretic098]

Thank you for the suggestion. I can confirm that changing config.ssl_verify: "False" to config.ssl_verify: "false" did not fix it.

However, I added this section to my helm values:

    env:
      lua_ssl_trusted_certificate: system
    extraSecrets:
      - name: keycloak-tls
        mountPath: /etc/ssl/cert.pem
        subPath: ca.crt

Which has fixed it. One slight wrinkle is that the keycloak-tls secret is in a different namespace so has to be copied to the kong-istio namespace for this to work. Of course it will all melt every three months when the certificate expires but that is manageable in a local development system.

Thanks again.

[heretic098]

Should the fact that the LUA doesn't respect the config.ssl_verify be a bug?

[ghost]

The default value of config.ssl_verify is false? and you can try not configuring it.