[BUG] Setup OpenID Connect with Latest Opensearch/Dashboards and Keycloak

[wantdrink]

Hi there,

I've installed opensearch + dashboards 2.3.0 with the latest chart. Also Keycloak bitnami/keycloak:19.0.2.
After configured as https://opensearch.org/docs/latest/security-plugin/configuration/openid-connect/#opensearch-dashboards-single-sign-on describes, and log in opensearch dashboard, it failed with:
{"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}

To Reproduce
Steps to reproduce the behavior:

1. Install Keycloak 19.0.2 with the latest Bitnami chart.
2. Create realm, user with password etc and generate user's secret in Keycloak.
3. Install Opensearch & Dashboards 2.3.0 with the latest chart and the secret created in step 2.
4. No SSL enabled, port-forward dashboard to 5601 and Keycloak to 8180 on K8S master node.
5. Try to access http://master:5601 and it failed with log

"res": {
    "statusCode": 401,
    "responseTime": 11,
    "contentLength": 9
},

Expected behavior
Login in Keycloak UI successfully and redirected to dashboards.

OpenSearch Version
2.3.0

Dashboards Version
2.3.0

Plugins
opensearch:

opensearch-alerting
opensearch-anomaly-detection
opensearch-asynchronous-search
opensearch-cross-cluster-replication
opensearch-geospatial
opensearch-index-management
opensearch-job-scheduler
opensearch-knn
opensearch-ml
opensearch-notifications
opensearch-notifications-core
opensearch-observability
opensearch-performance-analyzer
opensearch-reports-scheduler
opensearch-security
opensearch-sql

dashboards:

[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Configuration snippet in opensearch values:

config.yml: |-
 
  config:
        authc:
          basic_internal_auth_domain:
            http_enabled: true
            transport_enabled: true
            order: 0
            http_authenticator:
              type: basic
              challenge: false
            authentication_backend:
              type: internal

          openid_auth_domain:
            http_enabled: true
            transport_enabled: true
            order: 1
            http_authenticator:
              type: openid
              challenge: false
              config:
                openid_connect_idp:
                  enable_ssl: false
                  verify_hostnames: false
                  pemtrustedcas_filepath: /usr/share/opensearch/data/cert/cert
                subject_key: preferred_username
                roles_key: roles
                openid_connect_url: http://master.local.com:8180/realms/Myrealm/.well-known/openid-configuration
            authentication_backend:
              type: noop

Config snippet in dashboard values:

extraEnvs:
  - name: NODE_TLS_REJECT_UNAUTHORIZED
    value: "0"
config:
  opensearch_dashboards.yml:
    #server.basePath: "/"
    #server.rewriteBasePath: true
    server.host: "0.0.0.0"
    opensearch.hosts: [http://localhost:9200]
    opensearch.ssl.verificationMode: none
    opensearch.username: kibanaserver
    opensearch.password: kibanaserver
    opensearch.requestHeadersWhitelist: [authorization, securitytenant]
    opensearch_security.multitenancy.enabled: true
    opensearch_security.multitenancy.tenants.preferred: [Private, Global]
    opensearch_security.readonly_mode.roles: [opensearch_dashboards_read_only]
    # Use this setting if you are running opensearch-dashboards without https
    opensearch_security.cookie.secure: false
    opensearch_security.auth.type: "openid"
    opensearch_security.openid.connect_url: "http://master.local.com:8180/realms/Myrealm/.well-known/openid-configuration"
    opensearch_security.openid.client_id: "opensearch"
    opensearch_security.openid.client_secret: "Csf0NJlMP2aZY4jZpnGbhyB50b68tryC"
    opensearch_security.openid.verify_hostnames: false
    opensearch_security.openid.base_redirect_url: "http://master.local.com:5601"

Finally the log shows:

{
    "type": "response",
    "@timestamp": "2022-10-08T14:42:51Z",
    "tags": [],
    "pid": 1,
    "method": "get",
    "statusCode": 401,
    "req": {
        "url": "/favicon.ico",
        "method": "get",
        "headers": {
            "host": "master.local.com:5601",
            "connection": "keep-alive",
            "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.37",
            "accept": "image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8",
            "referer": "http://master.local.com:5601/auth/openid/login?state=rzGNjo2lmYs-E268tSEGvN&session_state=19ea485b-12ac-49ea-acd5-2d4907f86c2f&code=ec53282e-aa4c-42e9-a551-fced7448284e.19ea485b-12ac-49ea-acd5-2d4907f86c2f.db958e29-05c4-41b9-8337-e3837576eebc",
            "accept-encoding": "gzip, deflate",
            "accept-language": "en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7"
        },
        "remoteAddress": "127.0.0.1",
        "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.37",
        "referer": "http://master.local.com:5601/auth/openid/login?state=rzGNjo2lmYs-E268tSEGvN&session_state=19ea485b-12ac-49ea-acd5-2d4907f86c2f&code=ec53282e-aa4c-42e9-a551-fced7448284e.19ea485b-12ac-49ea-acd5-2d4907f86c2f.db958e29-05c4-41b9-8337-e3837576eebc"
    },
    "res": {
        "statusCode": 401,
        "responseTime": 11,
        "contentLength": 9
    },
    "message": "GET /favicon.ico 401 11ms - 9.0B"
}

BTW in https://opensearch.org/docs/latest/security-plugin/configuration/openid-connect/#opensearch-dashboards-single-sign-on those links in config are /auth/realms/xxx as below. I removed the /auth in the URL and not sure if that's a correct change in 2.3.0.

The IdP metadata endpoint

opensearch_security.openid.connect_url: "http://keycloak.example.com:8080/auth/realms/master/.well-known/openid-configuration"

Thanks.

[seraphjiang]

should we move this to https://github.com/opensearch-project/security-dashboards-plugin/

@cliu123 @aoguan1990

[aoguan1990]

@seraphjiang @cliu123 This is a security plugin configuration related issue. We can move it.

[cliu123]

@seraphjiang @aoguan1990
Yes, this should be transfered to security-dashboards-plugin.
Keycloak is not supported by the plugin at this moment. So not only configuration, enhancements will be needed.

@opensearch-project/transfer-request Please transfer the issue. Thanks!

[aoguan1990]

@cliu123 @seraphjiang Keycloak used standard OpenID Connect protocol, which is supported by our existing OIDC authentication handler. But we do need reproduce the issue based on information provided by the requester and confirm if this is a valid bug or configuration issue.

[seraphjiang]

@cliu123 @seraphjiang Keycloak used standard OpenID Connect protocol, which is supported by our existing OIDC authentication handler. But we do need reproduce the issue based on information provided by the requester and confirm if this is a valid bug or configuration issue.

Thanks @cliu123 and @aoguan1990 let's transfer to right repo and see what we could help

[cliu123]

@cliu123 @seraphjiang Keycloak used standard OpenID Connect protocol, which is supported by our existing OIDC authentication handler. But we do need reproduce the issue based on information provided by the requester and confirm if this is a valid bug or configuration issue.

@aoguan1990 is right. I mean Kerboros is not supported. OIDC is supported.

[linuxboyng]

@wantdrink I too had this problem (but with Okta OpenID) and resolved it with opensearch setting plugins.security.cache.ttl_minutes: 0 https://opensearch.org/docs/latest/security-plugin/configuration/configuration/#configure-json-web-tokens

What clued me onto this setting was I was able to get it to work in a docker-compose setup on the same machine but when I separated the dashboard to a different machine it failed to work and I kept getting the 401 error. Hope this helps.

[olddanmer]

Got the same with opensearch 2.3.0, dashboards 2.3.0, OpenID + Google IdP.
Setting plugins.security.cache.ttl_minutes: 0 helps, without it always have got error "statusCode":401,"error":"Unauthorized".

[aoguan1990]

@olddanmer I noticed that you enable anonymous_auth_enabled: true in your config.yml. Can you please provide more details for your authentication user cases?

[wantdrink]

Thanks @linuxboyng . Will try with the same configs.

[zalseryani]

Any update on this ?

I am unable to find any documentation on setting up opensearch with keycloak.

Thank you.

[RyanL1997]

Hi @wantdrink, based on what I observed, I think for the dashboards setting you have:

opensearch.hosts: [http://localhost:9200]

You may need to use https instead of http.

[davidlago]

Closing as potentially resolved by @RyanL1997 's suggestions since we have not heard back. Please feel free to re-open if not the case.