diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000000..3e4e2b7e9b --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,8 @@ +version: 2 +updates: + - package-ecosystem: "gradle" + directory: "/" + schedule: + interval: "weekly" + commit-message: + prefix: "dependabot:" diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 358b2eac14..9b0e9d62c4 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -10,6 +10,7 @@ So you want to contribute code to OpenSearch Security? Excellent! We're glad you - [Running integration tests](#running-integration-tests) - [Bulk test runs](#bulk-test-runs) - [Checkstyle Violations](#checkstyle-violations) + - [Authorization in REST Layer](#authorization-in-rest-layer) - [Submitting Changes](#submitting-changes) - [Backports](#backports) @@ -78,6 +79,51 @@ mv config/* $OPENSEARCH_HOME/config/opensearch-security/ rm -rf config/ ``` +### Installing demo extension users and roles + +If you are working with an extension and want to set up demo users for the Hello-World extension, append following items to files inside `$OPENSEARCH_HOME/config/opensearch-security/`: +1. In **internal_users.yml** +```yaml +hw-user: + hash: "$2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG" + reserved: true + description: "Demo user for ext-test" +``` + +2. In **roles.yml** +```yaml +extension_hw_greet: + reserved: true + cluster_permissions: + - 'hw:greet' + +extension_hw_full: + reserved: true + cluster_permissions: + - 'hw:goodbye' + - 'hw:greet' + - 'hw:greet_with_adjective' + - 'hw:greet_with_name' + +legacy_hw_greet_with_name: + reserved: true + cluster_permissions: + - 'cluster:admin/opensearch/hw/greet_with_name' +``` + +3. In **roles_mapping.yml** +```yaml +legacy_hw_greet_with_name: + reserved: true + users: + - "hw-user" + +extension_hw_greet: + reserved: true + users: + - "hw-user" +``` + To install the demo certificates and default configuration, answer `y` to the first two questions and `n` to the last one. The log should look like below: ```bash @@ -188,6 +234,11 @@ Checkstyle enforces several rules within this codebase. Sometimes it will be nec // CS-ENFORCE-ALL ``` +## Authorization in REST Layer + +See [REST_AUTHZ_FOR_PLUGINS](REST_AUTHZ_FOR_PLUGINS.md). + + ## Submitting Changes See [CONTRIBUTING](CONTRIBUTING.md). diff --git a/REST_AUTHZ_FOR_PLUGINS.md b/REST_AUTHZ_FOR_PLUGINS.md new file mode 100644 index 0000000000..b0f30ed04b --- /dev/null +++ b/REST_AUTHZ_FOR_PLUGINS.md @@ -0,0 +1,136 @@ +# Authorization at REST Layer for plugins + +This feature is introduced as an added layer of security on top of existing TransportLayer authorization framework. In order to leverage these feature some core changes need to be made at Route registration level. This document talks about how you can achieve this. + +**NOTE:** This doesn't replace Transport Layer Authorization. Plugin developers may choose to skip creating transport actions for APIs that do not need interaction with the Transport Layer. + +## Pre-requisites + +The security plugin must be installed and operational in your OpenSearch cluster for this feature to work. + +### How does NamedRoute authorization work? + +Once the routes are defined as NamedRoute, they, along-with their handlers, will be registered the same way as Route objects. When a request comes in, `SecurityRestFilter.java` applies an authorization check which extracts information about the NamedRoute. +Next we get the unique name and actionNames associated with that route and evaluate these against existing `cluster_permissions` across all roles of the requesting user. If the authorization check succeeds, the request chain proceeds as normal. If it fails, a 401 response is returned to the user. + +NOTE: +1. The action names defined in roles must exactly match the names of registered routes, or else, the request would be deemed unauthorized. +2. This check will not be implemented for plugins who do not use NamedRoutes. + + + +### How to translate an existing Route to be a NamedRoute? + +Here is a sample of an existing route converted to a named route: +Before: +``` +public List routes() { + return ImmutableList.of( + new Route(GET, "/uri") + ); +} +``` +With new scheme: +``` +public List routes() { + return ImmutableList.of( + new NamedRoute.Builder().method(GET).path("/uri").uniqueName("plugin:uri").actionNames(Set.of("cluster:admin/opensearch/plugin/uri")).build() + ); +} +``` + +`actionNames()` are optional. They correspond to any current actions defined as permissions in roles. +Ensure that these name-to-route mappings are easily accessible to the cluster admins to allow granting access to these APIs. + +### How does authorization in the REST Layer work? + +We will continue on the above example of translating `/uri` from Route to NamedRoute. + +Consider these roles are defined in the cluster: +```yaml +plugin_role: + reserved: true + cluster_permissions: + - 'plugin:uri' + +plugin_role_legacy: + reserved: true + cluster_permissions: + - 'cluster:admin/opensearch/plugin/uri' +``` + +Successful authz scenarios for a user: +1. The user is mapped either to `plugin_role` OR `plugin_role_legacy`. +2. The user is mapped to both of these roles. +3. The user is mapped to `plugin_role` even if no `actionNames()` were registered for this route. + +Unsuccessful authz scenarios for a user: +1. The user is not mapped any roles. +2. The user is mapped to a different role which doesn't grant the cluster permissions: `plugin:uri` OR `cluster:admin/opensearch/plugin/uri`/ +3. The user is mapped to a role `plugin_role_other` which has a typo in action name, i.e.`plugin:uuri`. + + +### Sample API in Security Plugin + +As part of this effort a new uri `GET /whoamiprotected` was introduced as a NamedRoute version of `GET /whoami`. Here is how you can test it: + +#### roles.yml +```yaml +who_am_i_role: + reserved: true + cluster_permissions: + - 'security:whoamiprotected' + +who_am_i_role_legacy: + reserved: true + cluster_permissions: + - 'cluster:admin/opendistro_security/whoamiprotected' + +who_am_i_role_no_perm: + reserved: true + cluster_permissions: + - 'some_invalid_perm' + +``` + +#### internal_users.yml +```yaml +who_am_i-user: + hash: "$2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG" #admin + reserved: true + description: "Demo user for ext-test" + +who_am_i_legacy-user: + hash: "$2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG" + reserved: true + description: "Demo user for ext-test" + +who_am_i_no_perm-user: + hash: "$2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG" + reserved: true + description: "Demo user for ext-test" +``` + +#### roles_mapping.yml +```yaml +who_am_i_role: + reserved: true + users: + - "who_am_i-user" + +who_am_i_role_legacy: + reserved: true + users: + - "who_am_i_legacy-user" + +who_am_i_role_no_perm: + reserved: true + users: + - "who_am_i_no_perm-user" +``` + +Follow [DEVELOPER_GUIDE](DEVELOPER_GUIDE.md) to setup OpenSearch cluster and initialize security plugin. Once you have verified that security plugin is installed correctly and OpenSearch is running, execute following curl requests: +1. `curl -XGET https://who_am_i-user:admin@localhost:9200/_plugins/_security/whoami --insecure` should succeed. +2. `curl -XGET https://who_am_i_legacy-user:admin@localhost:9200/_plugins/_security/whoami --insecure` should succeed. +3. `curl -XGET https://who_am_i_no-perm-user:admin@localhost:9200/_plugins/_security/whoami --insecure` should fail. +4. `curl -XPOST ` to `/whoami` with all 3 users should succeed. This is because POST route is not a NamedRoute and hence no authorization check was made. diff --git a/build.gradle b/build.gradle index 9233b01815..90888ac548 100644 --- a/build.gradle +++ b/build.gradle @@ -27,6 +27,11 @@ buildscript { common_utils_version = System.getProperty("common_utils.version", '3.0.0.0-SNAPSHOT') kafka_version = '3.5.0' apache_cxf_version = '4.0.2' + open_saml_version = '4.3.0' + one_login_java_saml = '2.9.0' + jjwt_version = '0.11.5' + guava_version = '32.1.1-jre' + jaxb_version = '2.3.8' if (buildVersionQualifier) { opensearch_build += "-${buildVersionQualifier}" @@ -37,11 +42,13 @@ buildscript { } repositories { - mavenCentral() mavenLocal() + mavenCentral() maven { url "https://plugins.gradle.org/m2/" } maven { url "https://aws.oss.sonatype.org/content/repositories/snapshots" } maven { url "https://d1nvenhzbhpy0q.cloudfront.net/snapshots/lucene/" } + maven { url "https://build.shibboleth.net/nexus/content/groups/public" } + maven { url "https://build.shibboleth.net/nexus/content/repositories/releases" } } dependencies { @@ -368,11 +375,12 @@ publishing { } repositories { - mavenCentral() mavenLocal() + mavenCentral() maven { url "https://plugins.gradle.org/m2/" } maven { url "https://aws.oss.sonatype.org/content/repositories/snapshots" } maven { url "https://d1nvenhzbhpy0q.cloudfront.net/snapshots/lucene/" } + maven { url "https://build.shibboleth.net/nexus/content/repositories/releases" } } tasks.withType(Checkstyle) { @@ -422,9 +430,10 @@ configurations { force "io.netty:netty-handler:${versions.netty}" force "io.netty:netty-transport:${versions.netty}" force "io.netty:netty-transport-native-unix-common:${versions.netty}" - force "org.apache.bcel:bcel:6.6.0" // This line should be removed once Spotbugs is upgraded to 4.7.4 + force "org.apache.bcel:bcel:6.7.0" // This line should be removed once Spotbugs is upgraded to 4.7.4 force "com.github.luben:zstd-jni:${versions.zstd}" force "org.xerial.snappy:snappy-java:1.1.10.1" + force "com.google.guava:guava:${guava_version}" } } @@ -476,18 +485,32 @@ dependencies { implementation "org.apache.httpcomponents:httpclient:${versions.httpclient}" implementation "org.apache.httpcomponents:httpcore:${versions.httpcore}" implementation "org.apache.httpcomponents:httpasyncclient:${versions.httpasyncclient}" - implementation 'com.google.guava:guava:32.0.1-jre' - implementation 'org.greenrobot:eventbus:3.2.0' + implementation "com.google.guava:guava:${guava_version}" + implementation 'org.greenrobot:eventbus-java:3.3.1' implementation 'commons-cli:commons-cli:1.5.0' implementation "org.bouncycastle:bcprov-jdk15to18:${versions.bouncycastle}" implementation 'org.ldaptive:ldaptive:1.2.3' - implementation 'io.jsonwebtoken:jjwt-api:0.10.8' - implementation 'com.github.wnameless:json-flattener:0.5.0' - implementation 'com.flipkart.zjsonpatch:zjsonpatch:0.4.4' + //JWT + implementation "io.jsonwebtoken:jjwt-api:${jjwt_version}" + implementation "io.jsonwebtoken:jjwt-impl:${jjwt_version}" + implementation "io.jsonwebtoken:jjwt-jackson:${jjwt_version}" + // JSON flattener + implementation ("com.github.wnameless.json:json-base:2.4.0") { + exclude group: "org.glassfish", module: "jakarta.json" + exclude group: "com.google.code.gson", module: "gson" + exclude group: "org.json", module: "json" + } + implementation 'com.github.wnameless.json:json-flattener:0.16.4' + // JSON patch + implementation 'com.flipkart.zjsonpatch:zjsonpatch:0.4.14' + implementation 'org.apache.commons:commons-collections4:4.4' + + //JSON path + implementation 'com.jayway.jsonpath:json-path:2.8.0' + implementation 'net.minidev:json-smart:2.4.11' + implementation "org.apache.kafka:kafka-clients:${kafka_version}" - implementation 'com.onelogin:java-saml:2.5.0' - implementation 'com.onelogin:java-saml-core:2.5.0' runtimeOnly 'net.minidev:accessors-smart:2.4.7' @@ -503,47 +526,45 @@ dependencies { runtimeOnly 'commons-codec:commons-codec:1.16.0' runtimeOnly 'org.cryptacular:cryptacular:1.2.4' runtimeOnly 'com.google.errorprone:error_prone_annotations:2.3.4' - runtimeOnly 'com.sun.istack:istack-commons-runtime:3.0.12' - runtimeOnly 'jakarta.xml.bind:jakarta.xml.bind-api:2.3.3' + runtimeOnly 'com.sun.istack:istack-commons-runtime:4.2.0' + runtimeOnly 'jakarta.xml.bind:jakarta.xml.bind-api:4.0.0' runtimeOnly 'org.ow2.asm:asm:9.1' testImplementation 'org.apache.camel:camel-xmlsecurity:3.14.2' - implementation 'net.shibboleth.utilities:java-support:7.5.1' - implementation 'org.opensaml:opensaml-core:3.4.5' - implementation 'org.opensaml:opensaml-security-impl:3.4.5' - implementation 'org.opensaml:opensaml-security-api:3.4.5' - implementation 'org.opensaml:opensaml-xmlsec-api:3.4.5' - implementation 'org.opensaml:opensaml-xmlsec-impl:3.4.5' - implementation 'org.opensaml:opensaml-saml-api:3.4.5' - implementation ('org.opensaml:opensaml-saml-impl:3.4.5') { + //OpenSAML + implementation 'net.shibboleth.utilities:java-support:8.4.0' + implementation "com.onelogin:java-saml:${one_login_java_saml}" + implementation "com.onelogin:java-saml-core:${one_login_java_saml}" + implementation "org.opensaml:opensaml-core:${open_saml_version}" + implementation "org.opensaml:opensaml-security-impl:${open_saml_version}" + implementation "org.opensaml:opensaml-security-api:${open_saml_version}" + implementation "org.opensaml:opensaml-xmlsec-api:${open_saml_version}" + implementation "org.opensaml:opensaml-xmlsec-impl:${open_saml_version}" + implementation "org.opensaml:opensaml-saml-api:${open_saml_version}" + implementation ("org.opensaml:opensaml-saml-impl:${open_saml_version}") { exclude(group: 'org.apache.velocity', module: 'velocity') } + implementation "org.opensaml:opensaml-messaging-api:${open_saml_version}" + runtimeOnly "org.opensaml:opensaml-profile-api:${open_saml_version}" + runtimeOnly "org.opensaml:opensaml-soap-api:${open_saml_version}" + runtimeOnly "org.opensaml:opensaml-soap-impl:${open_saml_version}" + implementation "org.opensaml:opensaml-storage-api:${open_saml_version}" + implementation "com.nulab-inc:zxcvbn:1.7.0" - testImplementation 'org.opensaml:opensaml-messaging-impl:3.4.5' - implementation 'org.opensaml:opensaml-messaging-api:3.4.5' - runtimeOnly 'org.opensaml:opensaml-profile-api:3.4.5' - runtimeOnly 'org.opensaml:opensaml-soap-api:3.4.5' - runtimeOnly 'org.opensaml:opensaml-soap-impl:3.4.5' - implementation 'org.opensaml:opensaml-storage-api:3.4.5' - implementation 'commons-collections:commons-collections:3.2.2' - implementation 'com.jayway.jsonpath:json-path:2.4.0' - implementation 'net.minidev:json-smart:2.4.10' - runtimeOnly 'io.jsonwebtoken:jjwt-impl:0.10.8' - runtimeOnly 'io.jsonwebtoken:jjwt-jackson:0.10.8' + runtimeOnly 'com.google.guava:failureaccess:1.0.1' runtimeOnly 'org.apache.commons:commons-text:1.10.0' - runtimeOnly 'org.glassfish.jaxb:jaxb-runtime:2.3.4' - runtimeOnly 'com.google.j2objc:j2objc-annotations:1.3' + runtimeOnly "org.glassfish.jaxb:jaxb-runtime:${jaxb_version}" + runtimeOnly 'com.google.j2objc:j2objc-annotations:2.8' runtimeOnly 'com.google.code.findbugs:jsr305:3.0.2' - runtimeOnly 'com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava' runtimeOnly 'org.lz4:lz4-java:1.8.0' runtimeOnly 'io.dropwizard.metrics:metrics-core:3.1.2' runtimeOnly 'org.slf4j:slf4j-api:1.7.30' - runtimeOnly 'org.apache.logging.log4j:log4j-slf4j-impl:2.17.1' + runtimeOnly "org.apache.logging.log4j:log4j-slf4j-impl:${versions.log4j}" runtimeOnly 'org.xerial.snappy:snappy-java:1.1.10.1' runtimeOnly 'org.codehaus.woodstox:stax2-api:4.2.1' - runtimeOnly 'org.glassfish.jaxb:txw2:2.3.4' + runtimeOnly "org.glassfish.jaxb:txw2:${jaxb_version}" runtimeOnly 'com.fasterxml.woodstox:woodstox-core:6.4.0' runtimeOnly 'org.apache.ws.xmlschema:xmlschema-core:2.2.5' runtimeOnly 'org.apache.santuario:xmlsec:2.2.3' @@ -553,6 +574,7 @@ dependencies { runtimeOnly 'org.scala-lang.modules:scala-java8-compat_3:1.0.2' + testImplementation "org.opensaml:opensaml-messaging-impl:${open_saml_version}" implementation 'org.apache.commons:commons-lang3:3.12.0' testImplementation "org.opensearch:common-utils:${common_utils_version}" testImplementation "org.opensearch.plugin:reindex-client:${opensearch_version}" @@ -561,7 +583,7 @@ dependencies { testImplementation "org.opensearch.plugin:lang-mustache-client:${opensearch_version}" testImplementation "org.opensearch.plugin:parent-join-client:${opensearch_version}" testImplementation "org.opensearch.plugin:aggs-matrix-stats-client:${opensearch_version}" - testImplementation 'org.apache.logging.log4j:log4j-core:2.17.1' + testImplementation "org.apache.logging.log4j:log4j-core:${versions.log4j}" testImplementation 'javax.servlet:servlet-api:2.5' testImplementation 'com.unboundid:unboundid-ldapsdk:4.0.9' testImplementation 'com.github.stephenc.jcip:jcip-annotations:1.0-1' @@ -583,7 +605,7 @@ dependencies { testImplementation "io.netty:netty-tcnative-boringssl-static:2.0.54.Final:${osdetector.classifier}" } // JUnit build requirement - testCompileOnly 'org.apiguardian:apiguardian-api:1.0.0' + testCompileOnly 'org.apiguardian:apiguardian-api:1.1.2' // Kafka test execution testRuntimeOnly 'org.springframework.retry:spring-retry:1.3.3' testRuntimeOnly ('org.springframework:spring-core:5.3.27') { @@ -609,8 +631,8 @@ dependencies { integrationTestImplementation "org.opensearch.plugin:reindex-client:${opensearch_version}" integrationTestImplementation "org.opensearch.plugin:percolator-client:${opensearch_version}" integrationTestImplementation 'commons-io:commons-io:2.11.0' - integrationTestImplementation 'org.apache.logging.log4j:log4j-core:2.17.1' - integrationTestImplementation 'org.apache.logging.log4j:log4j-jul:2.17.1' + integrationTestImplementation "org.apache.logging.log4j:log4j-core:${versions.log4j}" + integrationTestImplementation "org.apache.logging.log4j:log4j-jul:${versions.log4j}" integrationTestImplementation 'org.hamcrest:hamcrest:2.2' integrationTestImplementation "org.bouncycastle:bcpkix-jdk15to18:${versions.bouncycastle}" integrationTestImplementation "org.bouncycastle:bcutil-jdk15to18:${versions.bouncycastle}" @@ -618,6 +640,14 @@ dependencies { exclude(group: 'org.hamcrest', module: 'hamcrest') } integrationTestImplementation 'com.unboundid:unboundid-ldapsdk:4.0.9' + + //Checkstyle + checkstyle 'com.puppycrawl.tools:checkstyle:10.12.1' + + //spotless + implementation('com.google.googlejavaformat:google-java-format:1.17.0') { + exclude group: 'com.google.guava' + } } jar { diff --git a/bwc-test/gradle/wrapper/gradle-wrapper.jar b/bwc-test/gradle/wrapper/gradle-wrapper.jar index c1962a79e2..033e24c4cd 100644 Binary files a/bwc-test/gradle/wrapper/gradle-wrapper.jar and b/bwc-test/gradle/wrapper/gradle-wrapper.jar differ diff --git a/bwc-test/gradle/wrapper/gradle-wrapper.properties b/bwc-test/gradle/wrapper/gradle-wrapper.properties index 560c9869de..42d54bca55 100644 --- a/bwc-test/gradle/wrapper/gradle-wrapper.properties +++ b/bwc-test/gradle/wrapper/gradle-wrapper.properties @@ -1,7 +1,7 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists -distributionUrl=https\://services.gradle.org/distributions/gradle-8.1.1-bin.zip +distributionUrl=https\://services.gradle.org/distributions/gradle-8.2.1-bin.zip networkTimeout=10000 zipStoreBase=GRADLE_USER_HOME zipStorePath=wrapper/dists -distributionSha256Sum=e111cb9948407e26351227dabce49822fb88c37ee72f1d1582a69c68af2e702f +distributionSha256Sum=03ec176d388f2aa99defcadc3ac6adf8dd2bce5145a129659537c0874dea5ad1 diff --git a/bwc-test/gradlew b/bwc-test/gradlew index aeb74cbb43..fcb6fca147 100755 --- a/bwc-test/gradlew +++ b/bwc-test/gradlew @@ -130,10 +130,13 @@ location of your Java installation." fi else JAVACMD=java - which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. Please set the JAVA_HOME variable in your environment to match the location of your Java installation." + fi fi # Increase the maximum file descriptors if we can. diff --git a/config/roles.yml b/config/roles.yml index 05bdb6569b..3814a4fad4 100644 --- a/config/roles.yml +++ b/config/roles.yml @@ -257,6 +257,8 @@ cross_cluster_search_remote_full_access: ml_read_access: reserved: true cluster_permissions: + - 'cluster:admin/opensearch/ml/connectors/get' + - 'cluster:admin/opensearch/ml/connectors/search' - 'cluster:admin/opensearch/ml/model_groups/search' - 'cluster:admin/opensearch/ml/models/get' - 'cluster:admin/opensearch/ml/models/search' diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jar index c1962a79e2..033e24c4cd 100644 Binary files a/gradle/wrapper/gradle-wrapper.jar and b/gradle/wrapper/gradle-wrapper.jar differ diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties index 560c9869de..42d54bca55 100644 --- a/gradle/wrapper/gradle-wrapper.properties +++ b/gradle/wrapper/gradle-wrapper.properties @@ -1,7 +1,7 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists -distributionUrl=https\://services.gradle.org/distributions/gradle-8.1.1-bin.zip +distributionUrl=https\://services.gradle.org/distributions/gradle-8.2.1-bin.zip networkTimeout=10000 zipStoreBase=GRADLE_USER_HOME zipStorePath=wrapper/dists -distributionSha256Sum=e111cb9948407e26351227dabce49822fb88c37ee72f1d1582a69c68af2e702f +distributionSha256Sum=03ec176d388f2aa99defcadc3ac6adf8dd2bce5145a129659537c0874dea5ad1 diff --git a/gradlew b/gradlew index aeb74cbb43..fcb6fca147 100755 --- a/gradlew +++ b/gradlew @@ -130,10 +130,13 @@ location of your Java installation." fi else JAVACMD=java - which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. Please set the JAVA_HOME variable in your environment to match the location of your Java installation." + fi fi # Increase the maximum file descriptors if we can. diff --git a/plugin-security.policy b/plugin-security.policy index 7bb18f76c9..04643df0f3 100644 --- a/plugin-security.policy +++ b/plugin-security.policy @@ -60,7 +60,6 @@ grant { permission java.security.SecurityPermission "putProviderProperty.BC"; permission java.security.SecurityPermission "insertProvider.BC"; permission java.security.SecurityPermission "removeProviderProperty.BC"; - permission java.util.PropertyPermission "jdk.tls.rejectClientInitiatedRenegotiation", "write"; permission java.lang.RuntimePermission "accessUserInformation"; @@ -74,6 +73,10 @@ grant { //Enable this permission to debug unauthorized de-serialization attempt //permission java.io.SerializablePermission "enableSubstitution"; + + //SAML policy + permission java.util.PropertyPermission "*", "read,write"; + permission org.opensearch.secure_sm.ThreadPermission "modifyArbitraryThread"; }; grant codeBase "${codebase.netty-common}" { diff --git a/src/integrationTest/java/org/opensearch/security/api/DashboardsInfoTest.java b/src/integrationTest/java/org/opensearch/security/api/DashboardsInfoTest.java new file mode 100644 index 0000000000..a8936765d2 --- /dev/null +++ b/src/integrationTest/java/org/opensearch/security/api/DashboardsInfoTest.java @@ -0,0 +1,56 @@ +/* +* SPDX-License-Identifier: Apache-2.0 +* +* The OpenSearch Contributors require contributions made to +* this file be licensed under the Apache-2.0 license or a +* compatible open source license. +* +* Modifications Copyright OpenSearch Contributors. See +* GitHub history for details. +*/ + +package org.opensearch.security.api; + +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakScope; +import org.apache.hc.core5.http.HttpStatus; +import org.junit.ClassRule; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.opensearch.test.framework.TestSecurityConfig; +import org.opensearch.test.framework.TestSecurityConfig.Role; +import org.opensearch.test.framework.cluster.ClusterManager; +import org.opensearch.test.framework.cluster.LocalCluster; +import org.opensearch.test.framework.cluster.TestRestClient; + +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.containsString; +import static org.hamcrest.Matchers.equalTo; +import static org.opensearch.security.rest.DashboardsInfoAction.DEFAULT_PASSWORD_MESSAGE; +import static org.opensearch.test.framework.TestSecurityConfig.AuthcDomain.AUTHC_HTTPBASIC_INTERNAL; + +@RunWith(com.carrotsearch.randomizedtesting.RandomizedRunner.class) +@ThreadLeakScope(ThreadLeakScope.Scope.NONE) +public class DashboardsInfoTest { + + protected final static TestSecurityConfig.User DASHBOARDS_USER = new TestSecurityConfig.User("dashboards_user").roles( + new Role("dashboards_role").indexPermissions("read").on("*").clusterPermissions("cluster_composite_ops") + ); + + @ClassRule + public static LocalCluster cluster = new LocalCluster.Builder().clusterManager(ClusterManager.THREE_CLUSTER_MANAGERS) + .authc(AUTHC_HTTPBASIC_INTERNAL) + .users(DASHBOARDS_USER) + .build(); + + @Test + public void testDashboardsInfoValidationMessage() throws Exception { + + try (TestRestClient client = cluster.getRestClient(DASHBOARDS_USER)) { + TestRestClient.HttpResponse response = client.get("_plugins/_security/dashboardsinfo"); + assertThat(response.getStatusCode(), equalTo(HttpStatus.SC_OK)); + assertThat(response.getBody(), containsString("password_validation_error_message")); + assertThat(response.getBody(), containsString(DEFAULT_PASSWORD_MESSAGE)); + } + } +} diff --git a/src/integrationTest/java/org/opensearch/security/api/DashboardsInfoWithSettingsTest.java b/src/integrationTest/java/org/opensearch/security/api/DashboardsInfoWithSettingsTest.java new file mode 100644 index 0000000000..01654e17cd --- /dev/null +++ b/src/integrationTest/java/org/opensearch/security/api/DashboardsInfoWithSettingsTest.java @@ -0,0 +1,68 @@ +/* +* SPDX-License-Identifier: Apache-2.0 +* +* The OpenSearch Contributors require contributions made to +* this file be licensed under the Apache-2.0 license or a +* compatible open source license. +* +* Modifications Copyright OpenSearch Contributors. See +* GitHub history for details. +*/ + +package org.opensearch.security.api; + +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakScope; +import org.apache.hc.core5.http.HttpStatus; +import org.junit.ClassRule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.opensearch.security.support.ConfigConstants; +import org.opensearch.test.framework.TestSecurityConfig; +import org.opensearch.test.framework.TestSecurityConfig.Role; +import org.opensearch.test.framework.cluster.ClusterManager; +import org.opensearch.test.framework.cluster.LocalCluster; +import org.opensearch.test.framework.cluster.TestRestClient; + +import java.util.Map; + +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.containsString; +import static org.hamcrest.Matchers.equalTo; +import static org.opensearch.test.framework.TestSecurityConfig.AuthcDomain.AUTHC_HTTPBASIC_INTERNAL; + +@RunWith(com.carrotsearch.randomizedtesting.RandomizedRunner.class) +@ThreadLeakScope(ThreadLeakScope.Scope.NONE) +public class DashboardsInfoWithSettingsTest { + + protected final static TestSecurityConfig.User DASHBOARDS_USER = new TestSecurityConfig.User("dashboards_user").roles( + new Role("dashboards_role").indexPermissions("read").on("*").clusterPermissions("cluster_composite_ops") + ); + + private static final String CUSTOM_PASSWORD_MESSAGE = + "Password must be minimum 5 characters long and must contain at least one uppercase letter, one lowercase letter, one digit, and one special character."; + + @ClassRule + public static LocalCluster cluster = new LocalCluster.Builder().clusterManager(ClusterManager.THREE_CLUSTER_MANAGERS) + .authc(AUTHC_HTTPBASIC_INTERNAL) + .users(DASHBOARDS_USER) + .nodeSettings( + Map.of( + ConfigConstants.SECURITY_RESTAPI_PASSWORD_VALIDATION_REGEX, + "(?=.*[A-Z])(?=.*[^a-zA-Z\\d])(?=.*[0-9])(?=.*[a-z]).{5,}", + ConfigConstants.SECURITY_RESTAPI_PASSWORD_VALIDATION_ERROR_MESSAGE, + CUSTOM_PASSWORD_MESSAGE + ) + ) + .build(); + + @Test + public void testDashboardsInfoValidationMessageWithCustomMessage() throws Exception { + + try (TestRestClient client = cluster.getRestClient(DASHBOARDS_USER)) { + TestRestClient.HttpResponse response = client.get("_plugins/_security/dashboardsinfo"); + assertThat(response.getStatusCode(), equalTo(HttpStatus.SC_OK)); + assertThat(response.getBody(), containsString("password_validation_error_message")); + assertThat(response.getBody(), containsString(CUSTOM_PASSWORD_MESSAGE)); + } + } +} diff --git a/src/integrationTest/java/org/opensearch/security/rest/WhoAmITests.java b/src/integrationTest/java/org/opensearch/security/rest/WhoAmITests.java new file mode 100644 index 0000000000..5e9992c8ad --- /dev/null +++ b/src/integrationTest/java/org/opensearch/security/rest/WhoAmITests.java @@ -0,0 +1,107 @@ +/* +* SPDX-License-Identifier: Apache-2.0 +* +* The OpenSearch Contributors require contributions made to +* this file be licensed under the Apache-2.0 license or a +* compatible open source license. +* +* Modifications Copyright OpenSearch Contributors. See +* GitHub history for details. +*/ + +package org.opensearch.security.rest; + +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakScope; +import org.apache.hc.core5.http.HttpStatus; +import org.junit.ClassRule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.opensearch.test.framework.TestSecurityConfig; +import org.opensearch.test.framework.TestSecurityConfig.Role; +import org.opensearch.test.framework.cluster.ClusterManager; +import org.opensearch.test.framework.cluster.LocalCluster; +import org.opensearch.test.framework.cluster.TestRestClient; + +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.equalTo; +import static org.opensearch.test.framework.TestSecurityConfig.AuthcDomain.AUTHC_HTTPBASIC_INTERNAL; + +@RunWith(com.carrotsearch.randomizedtesting.RandomizedRunner.class) +@ThreadLeakScope(ThreadLeakScope.Scope.NONE) +public class WhoAmITests { + protected final static TestSecurityConfig.User WHO_AM_I = new TestSecurityConfig.User("who_am_i_user").roles( + new Role("who_am_i_role").clusterPermissions("security:whoamiprotected") + ); + + protected final static TestSecurityConfig.User WHO_AM_I_LEGACY = new TestSecurityConfig.User("who_am_i_user_legacy").roles( + new Role("who_am_i_role_legacy").clusterPermissions("cluster:admin/opendistro_security/whoamiprotected") + ); + + protected final static TestSecurityConfig.User WHO_AM_I_NO_PERM = new TestSecurityConfig.User("who_am_i_user_no_perm").roles( + new Role("who_am_i_role_no_perm") + ); + + protected final static TestSecurityConfig.User WHO_AM_I_UNREGISTERED = new TestSecurityConfig.User("who_am_i_user_no_perm"); + + public static final String WHOAMI_ENDPOINT = "_plugins/_security/whoami"; + public static final String WHOAMI_PROTECTED_ENDPOINT = "_plugins/_security/whoamiprotected"; + + @ClassRule + public static LocalCluster cluster = new LocalCluster.Builder().clusterManager(ClusterManager.THREE_CLUSTER_MANAGERS) + .authc(AUTHC_HTTPBASIC_INTERNAL) + .users(WHO_AM_I, WHO_AM_I_LEGACY, WHO_AM_I_NO_PERM) + .build(); + + @Test + public void testWhoAmIWithGetPermissions() throws Exception { + try (TestRestClient client = cluster.getRestClient(WHO_AM_I)) { + assertThat(client.get(WHOAMI_PROTECTED_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + + try (TestRestClient client = cluster.getRestClient(WHO_AM_I)) { + assertThat(client.get(WHOAMI_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + } + + @Test + public void testWhoAmIWithGetPermissionsLegacy() throws Exception { + try (TestRestClient client = cluster.getRestClient(WHO_AM_I_LEGACY)) { + assertThat(client.get(WHOAMI_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + + try (TestRestClient client = cluster.getRestClient(WHO_AM_I_LEGACY)) { + assertThat(client.get(WHOAMI_PROTECTED_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + } + + @Test + public void testWhoAmIWithoutGetPermissions() throws Exception { + try (TestRestClient client = cluster.getRestClient(WHO_AM_I_NO_PERM)) { + assertThat(client.get(WHOAMI_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + + try (TestRestClient client = cluster.getRestClient(WHO_AM_I_NO_PERM)) { + assertThat(client.get(WHOAMI_PROTECTED_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_UNAUTHORIZED)); + } + } + + @Test + public void testWhoAmIPost() throws Exception { + try (TestRestClient client = cluster.getRestClient(WHO_AM_I)) { + assertThat(client.post(WHOAMI_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + + try (TestRestClient client = cluster.getRestClient(WHO_AM_I_LEGACY)) { + assertThat(client.post(WHOAMI_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + + try (TestRestClient client = cluster.getRestClient(WHO_AM_I_NO_PERM)) { + assertThat(client.post(WHOAMI_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + + try (TestRestClient client = cluster.getRestClient(WHO_AM_I_UNREGISTERED)) { + assertThat(client.post(WHOAMI_ENDPOINT).getStatusCode(), equalTo(HttpStatus.SC_OK)); + } + + } +} diff --git a/src/integrationTest/java/org/opensearch/test/framework/log/LogCapturingAppender.java b/src/integrationTest/java/org/opensearch/test/framework/log/LogCapturingAppender.java index 63765dfd14..5673f1bd3e 100644 --- a/src/integrationTest/java/org/opensearch/test/framework/log/LogCapturingAppender.java +++ b/src/integrationTest/java/org/opensearch/test/framework/log/LogCapturingAppender.java @@ -9,18 +9,8 @@ */ package org.opensearch.test.framework.log; -import java.io.Serializable; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Collections; -import java.util.HashSet; -import java.util.List; -import java.util.Set; -import java.util.stream.Collectors; - -import org.apache.commons.collections.Buffer; -import org.apache.commons.collections.BufferUtils; -import org.apache.commons.collections.buffer.CircularFifoBuffer; +import com.google.common.collect.EvictingQueue; +import com.google.common.collect.Queues; import org.apache.logging.log4j.core.Appender; import org.apache.logging.log4j.core.Core; import org.apache.logging.log4j.core.Filter; @@ -32,6 +22,15 @@ import org.apache.logging.log4j.core.config.plugins.PluginAttribute; import org.apache.logging.log4j.core.config.plugins.PluginFactory; +import java.io.Serializable; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Queue; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; +import java.util.stream.Collectors; + import static org.opensearch.test.framework.log.LogCapturingAppender.PLUGIN_NAME; /** @@ -56,12 +55,12 @@ public class LogCapturingAppender extends AbstractAppender { /** * Buffer for captured log messages */ - private static final Buffer messages = BufferUtils.synchronizedBuffer(new CircularFifoBuffer(MAX_SIZE)); + private static final Queue messages = Queues.synchronizedQueue(EvictingQueue.create(MAX_SIZE)); /** * Log messages are stored in buffer {@link #messages} only for classes which are added to the {@link #activeLoggers} set. */ - private static final Set activeLoggers = Collections.synchronizedSet(new HashSet<>()); + private static final Set activeLoggers = ConcurrentHashMap.newKeySet(); protected LogCapturingAppender( String name, diff --git a/src/main/java/com/amazon/dlic/auth/http/saml/AuthTokenProcessorHandler.java b/src/main/java/com/amazon/dlic/auth/http/saml/AuthTokenProcessorHandler.java index 1c49d10b2e..7f36635674 100644 --- a/src/main/java/com/amazon/dlic/auth/http/saml/AuthTokenProcessorHandler.java +++ b/src/main/java/com/amazon/dlic/auth/http/saml/AuthTokenProcessorHandler.java @@ -168,9 +168,7 @@ private AuthTokenProcessorAction.Response handleImpl( try { - SamlResponse samlResponse = new SamlResponse(saml2Settings, null); - samlResponse.setDestinationUrl(acsEndpoint); - samlResponse.loadXmlFromBase64(samlResponseBase64); + final SamlResponse samlResponse = new SamlResponse(saml2Settings, acsEndpoint, samlResponseBase64); if (!samlResponse.isValid(samlRequestId)) { log.warn("Error while validating SAML response in /_opendistro/_security/api/authtoken"); diff --git a/src/main/java/com/amazon/dlic/auth/http/saml/Saml2SettingsProvider.java b/src/main/java/com/amazon/dlic/auth/http/saml/Saml2SettingsProvider.java index 881c9c3553..1b97242762 100644 --- a/src/main/java/com/amazon/dlic/auth/http/saml/Saml2SettingsProvider.java +++ b/src/main/java/com/amazon/dlic/auth/http/saml/Saml2SettingsProvider.java @@ -14,6 +14,7 @@ import java.security.AccessController; import java.security.PrivateKey; import java.security.PrivilegedAction; +import java.time.Instant; import java.util.AbstractMap; import java.util.Collection; import java.util.HashMap; @@ -28,7 +29,6 @@ import net.shibboleth.utilities.java.support.resolver.ResolverException; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; -import org.joda.time.DateTime; import org.opensaml.core.criterion.EntityIdCriterion; import org.opensaml.saml.metadata.resolver.MetadataResolver; import org.opensaml.saml.metadata.resolver.RefreshableMetadataResolver; @@ -54,7 +54,7 @@ public class Saml2SettingsProvider { private final String idpEntityId; private final PrivateKey spSignaturePrivateKey; private Saml2Settings cachedSaml2Settings; - private DateTime metadataUpdateTime; + private Instant metadataUpdateTime; Saml2SettingsProvider(Settings opensearchSettings, MetadataResolver metadataResolver, PrivateKey spSignaturePrivateKey) { this.opensearchSettings = opensearchSettings; @@ -107,7 +107,7 @@ Saml2Settings get() throws SamlConfigException { } Saml2Settings getCached() throws SamlConfigException { - DateTime tempLastUpdate = null; + Instant tempLastUpdate = null; if (this.metadataResolver instanceof RefreshableMetadataResolver && this.isUpdateRequired()) { this.cachedSaml2Settings = null; diff --git a/src/main/java/com/amazon/dlic/auth/http/saml/SamlHTTPMetadataResolver.java b/src/main/java/com/amazon/dlic/auth/http/saml/SamlHTTPMetadataResolver.java index 015e8df12d..2a380539e6 100644 --- a/src/main/java/com/amazon/dlic/auth/http/saml/SamlHTTPMetadataResolver.java +++ b/src/main/java/com/amazon/dlic/auth/http/saml/SamlHTTPMetadataResolver.java @@ -15,6 +15,7 @@ import java.security.AccessController; import java.security.PrivilegedActionException; import java.security.PrivilegedExceptionAction; +import java.time.Duration; import net.shibboleth.utilities.java.support.resolver.ResolverException; import org.apache.http.client.HttpClient; @@ -31,8 +32,8 @@ public class SamlHTTPMetadataResolver extends HTTPMetadataResolver { SamlHTTPMetadataResolver(String idpMetadataUrl, Settings opensearchSettings, Path configPath) throws Exception { super(createHttpClient(opensearchSettings, configPath), idpMetadataUrl); - setMinRefreshDelay(opensearchSettings.getAsLong("idp.min_refresh_delay", 60L * 1000L)); - setMaxRefreshDelay(opensearchSettings.getAsLong("idp.max_refresh_delay", 14400000L)); + setMinRefreshDelay(Duration.ofMillis(opensearchSettings.getAsLong("idp.min_refresh_delay", 60L * 1000L))); + setMaxRefreshDelay(Duration.ofMillis(opensearchSettings.getAsLong("idp.max_refresh_delay", 14400000L))); setRefreshDelayFactor(opensearchSettings.getAsFloat("idp.refresh_delay_factor", 0.75f)); } diff --git a/src/main/java/org/opensearch/security/OpenSearchSecurityPlugin.java b/src/main/java/org/opensearch/security/OpenSearchSecurityPlugin.java index c32f4c7078..2568d3cc37 100644 --- a/src/main/java/org/opensearch/security/OpenSearchSecurityPlugin.java +++ b/src/main/java/org/opensearch/security/OpenSearchSecurityPlugin.java @@ -73,6 +73,7 @@ import org.opensearch.action.support.ActionFilter; import org.opensearch.client.Client; import org.opensearch.cluster.metadata.IndexNameExpressionResolver; +import org.opensearch.cluster.node.DiscoveryNode; import org.opensearch.cluster.node.DiscoveryNodes; import org.opensearch.cluster.service.ClusterService; import org.opensearch.common.component.Lifecycle.State; @@ -147,6 +148,7 @@ import org.opensearch.security.http.XFFResolver; import org.opensearch.security.privileges.PrivilegesEvaluator; import org.opensearch.security.privileges.PrivilegesInterceptor; +import org.opensearch.security.privileges.RestLayerPrivilegesEvaluator; import org.opensearch.security.resolver.IndexResolverReplacer; import org.opensearch.security.rest.DashboardsInfoAction; import org.opensearch.security.rest.SecurityConfigUpdateAction; @@ -206,10 +208,12 @@ public final class OpenSearchSecurityPlugin extends OpenSearchSecuritySSLPlugin private volatile SecurityInterceptor si; private volatile PrivilegesEvaluator evaluator; private volatile UserService userService; + private volatile RestLayerPrivilegesEvaluator restLayerEvaluator; private volatile ThreadPool threadPool; private volatile ConfigurationRepository cr; private volatile AdminDNs adminDns; private volatile ClusterService cs; + private static volatile DiscoveryNode localNode; private volatile AuditLog auditLog; private volatile BackendRegistry backendRegistry; private volatile SslExceptionHandler sslExceptionHandler; @@ -1024,8 +1028,11 @@ public Collection createComponents( principalExtractor = ReflectionHelper.instantiatePrincipalExtractor(principalExtractorClass); } + restLayerEvaluator = new RestLayerPrivilegesEvaluator(clusterService, threadPool, auditLog, cih, namedXContentRegistry); + securityRestHandler = new SecurityRestFilter( backendRegistry, + restLayerEvaluator, auditLog, threadPool, principalExtractor, @@ -1039,6 +1046,7 @@ public Collection createComponents( dcf.registerDCFListener(irr); dcf.registerDCFListener(xffResolver); dcf.registerDCFListener(evaluator); + dcf.registerDCFListener(restLayerEvaluator); dcf.registerDCFListener(securityRestHandler); if (!(auditLog instanceof NullAuditLog)) { // Don't register if advanced modules is disabled in which case auditlog is instance of NullAuditLog @@ -1076,6 +1084,7 @@ public Collection createComponents( components.add(xffResolver); components.add(backendRegistry); components.add(evaluator); + components.add(restLayerEvaluator); components.add(si); components.add(dcf); components.add(userService); @@ -1796,11 +1805,12 @@ public List getSettingsFilter() { } @Override - public void onNodeStarted() { + public void onNodeStarted(DiscoveryNode localNode) { log.info("Node started"); if (!SSLConfig.isSslOnlyMode() && !client && !disabled) { cr.initOnNodeStart(); } + this.localNode = localNode; final Set securityModules = ReflectionHelper.getModulesLoaded(); log.info("{} OpenSearch Security modules loaded so far: {}", securityModules.size(), securityModules); } @@ -1880,6 +1890,14 @@ private static String handleKeyword(final String field) { return field; } + public static DiscoveryNode getLocalNode() { + return localNode; + } + + public static void setLocalNode(DiscoveryNode node) { + localNode = node; + } + public static class GuiceHolder implements LifecycleComponent { private static RepositoriesService repositoriesService; diff --git a/src/main/java/org/opensearch/security/auditlog/impl/AbstractAuditLog.java b/src/main/java/org/opensearch/security/auditlog/impl/AbstractAuditLog.java index e14d5b17a9..fe5b01fec7 100644 --- a/src/main/java/org/opensearch/security/auditlog/impl/AbstractAuditLog.java +++ b/src/main/java/org/opensearch/security/auditlog/impl/AbstractAuditLog.java @@ -184,6 +184,7 @@ public void logMissingPrivileges(String privilege, String effectiveUser, RestReq msg.addRemoteAddress(remoteAddress); msg.addRestRequestInfo(request, auditConfigFilter); msg.addEffectiveUser(effectiveUser); + msg.addPrivilege(privilege); save(msg); } diff --git a/src/main/java/org/opensearch/security/compliance/FieldReadCallback.java b/src/main/java/org/opensearch/security/compliance/FieldReadCallback.java index 73f536c2f8..5fc1c73128 100644 --- a/src/main/java/org/opensearch/security/compliance/FieldReadCallback.java +++ b/src/main/java/org/opensearch/security/compliance/FieldReadCallback.java @@ -105,7 +105,7 @@ public void binaryFieldRead(final FieldInfo fieldInfo, byte[] fieldValue) { fieldValue = Utils.jsonMapToByteArray(filteredSource); } - Map filteredSource = new JsonFlattener(new String(fieldValue, StandardCharsets.UTF_8)).flattenAsMap(); + final Map filteredSource = JsonFlattener.flattenAsMap(new String(fieldValue, StandardCharsets.UTF_8)); for (String k : filteredSource.keySet()) { if (!recordField(k, filteredSource.get(k) instanceof String)) { continue; diff --git a/src/main/java/org/opensearch/security/dlic/rest/support/Utils.java b/src/main/java/org/opensearch/security/dlic/rest/support/Utils.java index 74908dbf60..65524a8bf7 100644 --- a/src/main/java/org/opensearch/security/dlic/rest/support/Utils.java +++ b/src/main/java/org/opensearch/security/dlic/rest/support/Utils.java @@ -44,6 +44,7 @@ import org.opensearch.core.xcontent.NamedXContentRegistry; import org.opensearch.core.xcontent.ToXContent; import org.opensearch.core.xcontent.XContentParser; +import org.opensearch.rest.NamedRoute; import org.opensearch.rest.RestHandler.DeprecatedRoute; import org.opensearch.rest.RestHandler.Route; import org.opensearch.security.DefaultObjectMapper; @@ -240,9 +241,17 @@ public static List addRoutesPrefix(List routes) { * Total number of routes will be expanded len(prefixes) as much comparing to the list passed in */ public static List addRoutesPrefix(List routes, final String... prefixes) { - return routes.stream() - .flatMap(r -> Arrays.stream(prefixes).map(p -> new Route(r.getMethod(), p + r.getPath()))) - .collect(ImmutableList.toImmutableList()); + return routes.stream().flatMap(r -> Arrays.stream(prefixes).map(p -> { + if (r instanceof NamedRoute) { + NamedRoute nr = (NamedRoute) r; + return new NamedRoute.Builder().method(nr.getMethod()) + .path(p + nr.getPath()) + .uniqueName(nr.name()) + .legacyActionNames(nr.actionNames()) + .build(); + } + return new Route(r.getMethod(), p + r.getPath()); + })).collect(ImmutableList.toImmutableList()); } /** diff --git a/src/main/java/org/opensearch/security/filter/SecurityRestFilter.java b/src/main/java/org/opensearch/security/filter/SecurityRestFilter.java index 80bba54ea2..cfc157d334 100644 --- a/src/main/java/org/opensearch/security/filter/SecurityRestFilter.java +++ b/src/main/java/org/opensearch/security/filter/SecurityRestFilter.java @@ -27,6 +27,9 @@ package org.opensearch.security.filter; import java.nio.file.Path; +import java.util.List; +import java.util.Optional; +import java.util.Set; import java.util.regex.Matcher; import java.util.regex.Pattern; @@ -37,10 +40,10 @@ import org.greenrobot.eventbus.Subscribe; import org.opensearch.OpenSearchException; -import org.opensearch.client.node.NodeClient; import org.opensearch.common.settings.Settings; import org.opensearch.common.util.concurrent.ThreadContext; import org.opensearch.rest.BytesRestResponse; +import org.opensearch.rest.NamedRoute; import org.opensearch.rest.RestChannel; import org.opensearch.rest.RestHandler; import org.opensearch.rest.RestRequest; @@ -52,6 +55,8 @@ import org.opensearch.security.configuration.AdminDNs; import org.opensearch.security.configuration.CompatConfig; import org.opensearch.security.dlic.rest.api.AllowlistApiAction; +import org.opensearch.security.privileges.PrivilegesEvaluatorResponse; +import org.opensearch.security.privileges.RestLayerPrivilegesEvaluator; import org.opensearch.security.securityconf.impl.AllowlistingSettings; import org.opensearch.security.securityconf.impl.WhitelistingSettings; import org.opensearch.security.ssl.transport.PrincipalExtractor; @@ -70,6 +75,7 @@ public class SecurityRestFilter { protected final Logger log = LogManager.getLogger(this.getClass()); private final BackendRegistry registry; + private final RestLayerPrivilegesEvaluator evaluator; private final AuditLog auditLog; private final ThreadContext threadContext; private final PrincipalExtractor principalExtractor; @@ -88,6 +94,7 @@ public class SecurityRestFilter { public SecurityRestFilter( final BackendRegistry registry, + final RestLayerPrivilegesEvaluator evaluator, final AuditLog auditLog, final ThreadPool threadPool, final PrincipalExtractor principalExtractor, @@ -97,6 +104,7 @@ public SecurityRestFilter( ) { super(); this.registry = registry; + this.evaluator = evaluator; this.auditLog = auditLog; this.threadContext = threadPool.getThreadContext(); this.principalExtractor = principalExtractor; @@ -109,28 +117,27 @@ public SecurityRestFilter( /** * This function wraps around all rest requests - * If the request is authenticated, then it goes through a whitelisting check. - * The whitelisting check works as follows: - * If whitelisting is not enabled, then requests are handled normally. - * If whitelisting is enabled, then SuperAdmin is allowed access to all APIs, regardless of what is currently whitelisted. - * If whitelisting is enabled, then Non-SuperAdmin is allowed to access only those APIs that are whitelisted in {@link #requests} - * For example: if whitelisting is enabled and requests = ["/_cat/nodes"], then SuperAdmin can access all APIs, but non SuperAdmin + * If the request is authenticated, then it goes through a allowlisting check. + * The allowlisting check works as follows: + * If allowlisting is not enabled, then requests are handled normally. + * If allowlisting is enabled, then SuperAdmin is allowed access to all APIs, regardless of what is currently allowlisted. + * If allowlisting is enabled, then Non-SuperAdmin is allowed to access only those APIs that are allowlisted in {@link #requests} + * For example: if allowlisting is enabled and requests = ["/_cat/nodes"], then SuperAdmin can access all APIs, but non SuperAdmin * can only access "/_cat/nodes" - * Further note: Some APIs are only accessible by SuperAdmin, regardless of whitelisting. For example: /_opendistro/_security/api/whitelist is only accessible by SuperAdmin. + * Further note: Some APIs are only accessible by SuperAdmin, regardless of allowlisting. For example: /_opendistro/_security/api/whitelist is only accessible by SuperAdmin. * See {@link AllowlistApiAction} for the implementation of this API. * SuperAdmin is identified by credentials, which can be passed in the curl request. */ public RestHandler wrap(RestHandler original, AdminDNs adminDNs) { - return new RestHandler() { - - @Override - public void handleRequest(RestRequest request, RestChannel channel, NodeClient client) throws Exception { - org.apache.logging.log4j.ThreadContext.clearAll(); - if (!checkAndAuthenticateRequest(request, channel, client)) { - User user = threadContext.getTransient(ConfigConstants.OPENDISTRO_SECURITY_USER); - if (userIsSuperAdmin(user, adminDNs) - || (whitelistingSettings.checkRequestIsAllowed(request, channel, client) - && allowlistingSettings.checkRequestIsAllowed(request, channel, client))) { + return (request, channel, client) -> { + org.apache.logging.log4j.ThreadContext.clearAll(); + if (!checkAndAuthenticateRequest(request, channel)) { + User user = threadContext.getTransient(ConfigConstants.OPENDISTRO_SECURITY_USER); + boolean isSuperAdminUser = userIsSuperAdmin(user, adminDNs); + if (isSuperAdminUser + || (whitelistingSettings.checkRequestIsAllowed(request, channel, client) + && allowlistingSettings.checkRequestIsAllowed(request, channel, client))) { + if (isSuperAdminUser || authorizeRequest(original, request, channel, user)) { original.handleRequest(request, channel, client); } } @@ -145,7 +152,54 @@ private boolean userIsSuperAdmin(User user, AdminDNs adminDNs) { return user != null && adminDNs.isAdmin(user); } - private boolean checkAndAuthenticateRequest(RestRequest request, RestChannel channel, NodeClient client) throws Exception { + private boolean authorizeRequest(RestHandler original, RestRequest request, RestChannel channel, User user) { + + List restRoutes = original.routes(); + Optional handler = restRoutes.stream() + .filter(rh -> rh.getMethod().equals(request.method())) + .filter(rh -> restPathMatches(request.path(), rh.getPath())) + .findFirst(); + final boolean routeSupportsRestAuthorization = handler.isPresent() && handler.get() instanceof NamedRoute; + if (routeSupportsRestAuthorization) { + PrivilegesEvaluatorResponse pres = new PrivilegesEvaluatorResponse(); + NamedRoute route = ((NamedRoute) handler.get()); + // if actionNames are present evaluate those first + Set actionNames = route.actionNames(); + if (actionNames != null && !actionNames.isEmpty()) { + pres = evaluator.evaluate(user, actionNames); + } + + // now if pres.allowed is still false check for the NamedRoute name as a permission + if (!pres.isAllowed()) { + String action = route.name(); + pres = evaluator.evaluate(user, Set.of(action)); + } + + if (log.isDebugEnabled()) { + log.debug(pres.toString()); + } + if (pres.isAllowed()) { + log.debug("Request has been granted"); + auditLog.logGrantedPrivileges(user.getName(), request); + } else { + auditLog.logMissingPrivileges(route.name(), user.getName(), request); + String err; + if (!pres.getMissingSecurityRoles().isEmpty()) { + err = String.format("No mapping for %s on roles %s", user, pres.getMissingSecurityRoles()); + } else { + err = String.format("no permissions for %s and %s", pres.getMissingPrivileges(), user); + } + log.debug(err); + channel.sendResponse(new BytesRestResponse(RestStatus.UNAUTHORIZED, err)); + return false; + } + } + + // if handler is not an instance of NamedRoute then we pass through to eval at Transport Layer. + return true; + } + + private boolean checkAndAuthenticateRequest(RestRequest request, RestChannel channel) throws Exception { threadContext.putTransient(ConfigConstants.OPENDISTRO_SECURITY_ORIGIN, Origin.REST.toString()); @@ -217,4 +271,30 @@ public void onWhitelistingSettingChanged(WhitelistingSettings whitelistingSettin public void onAllowlistingSettingChanged(AllowlistingSettings allowlistingSettings) { this.allowlistingSettings = allowlistingSettings; } + + /** + * Determines if the request's path is a match for the configured handler path. + * + * @param requestPath The path from the {@link NamedRoute} + * @param handlerPath The path from the {@link RestHandler.Route} + * @return true if the request path matches the route + */ + private boolean restPathMatches(String requestPath, String handlerPath) { + // Check exact match + if (handlerPath.equals(requestPath)) { + return true; + } + // Split path to evaluate named params + String[] handlerSplit = handlerPath.split("/"); + String[] requestSplit = requestPath.split("/"); + if (handlerSplit.length != requestSplit.length) { + return false; + } + for (int i = 0; i < handlerSplit.length; i++) { + if (!(handlerSplit[i].equals(requestSplit[i]) || (handlerSplit[i].startsWith("{") && handlerSplit[i].endsWith("}")))) { + return false; + } + } + return true; + } } diff --git a/src/main/java/org/opensearch/security/privileges/RestLayerPrivilegesEvaluator.java b/src/main/java/org/opensearch/security/privileges/RestLayerPrivilegesEvaluator.java new file mode 100644 index 0000000000..301207022b --- /dev/null +++ b/src/main/java/org/opensearch/security/privileges/RestLayerPrivilegesEvaluator.java @@ -0,0 +1,130 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.security.privileges; + +import java.util.Set; +import java.util.concurrent.atomic.AtomicReference; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.greenrobot.eventbus.Subscribe; + +import org.opensearch.OpenSearchSecurityException; +import org.opensearch.cluster.service.ClusterService; +import org.opensearch.common.transport.TransportAddress; +import org.opensearch.common.util.concurrent.ThreadContext; +import org.opensearch.core.xcontent.NamedXContentRegistry; +import org.opensearch.security.auditlog.AuditLog; +import org.opensearch.security.configuration.ClusterInfoHolder; +import org.opensearch.security.securityconf.ConfigModel; +import org.opensearch.security.securityconf.DynamicConfigModel; +import org.opensearch.security.securityconf.SecurityRoles; +import org.opensearch.security.support.ConfigConstants; +import org.opensearch.security.user.User; +import org.opensearch.threadpool.ThreadPool; + +public class RestLayerPrivilegesEvaluator { + protected final Logger log = LogManager.getLogger(this.getClass()); + private final ClusterService clusterService; + private final AuditLog auditLog; + private ThreadContext threadContext; + private final ClusterInfoHolder clusterInfoHolder; + private ConfigModel configModel; + private DynamicConfigModel dcm; + private final AtomicReference namedXContentRegistry; + + public RestLayerPrivilegesEvaluator( + final ClusterService clusterService, + final ThreadPool threadPool, + AuditLog auditLog, + final ClusterInfoHolder clusterInfoHolder, + AtomicReference namedXContentRegistry + ) { + this.clusterService = clusterService; + this.auditLog = auditLog; + + this.threadContext = threadPool.getThreadContext(); + + this.clusterInfoHolder = clusterInfoHolder; + this.namedXContentRegistry = namedXContentRegistry; + } + + @Subscribe + public void onConfigModelChanged(ConfigModel configModel) { + this.configModel = configModel; + } + + @Subscribe + public void onDynamicConfigModelChanged(DynamicConfigModel dcm) { + this.dcm = dcm; + } + + private SecurityRoles getSecurityRoles(Set roles) { + return configModel.getSecurityRoles().filter(roles); + } + + public boolean isInitialized() { + return configModel != null && configModel.getSecurityRoles() != null && dcm != null; + } + + public PrivilegesEvaluatorResponse evaluate(final User user, Set actions) { + + if (!isInitialized()) { + throw new OpenSearchSecurityException("OpenSearch Security is not initialized."); + } + + final PrivilegesEvaluatorResponse presponse = new PrivilegesEvaluatorResponse(); + + final TransportAddress caller = threadContext.getTransient(ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS); + + Set mappedRoles = mapRoles(user, caller); + + presponse.resolvedSecurityRoles.addAll(mappedRoles); + final SecurityRoles securityRoles = getSecurityRoles(mappedRoles); + + final boolean isDebugEnabled = log.isDebugEnabled(); + if (isDebugEnabled) { + log.debug("Evaluate permissions for {} on {}", user, clusterService.localNode().getName()); + log.debug("Action: {}", actions); + log.debug("Mapped roles: {}", mappedRoles.toString()); + } + + for (String action : actions) { + if (!securityRoles.impliesClusterPermissionPermission(action)) { + presponse.missingPrivileges.add(action); + presponse.allowed = false; + log.info( + "No permission match for {} [Action [{}]] [RolesChecked {}]. No permissions for {}", + user, + action, + securityRoles.getRoleNames(), + presponse.missingPrivileges + ); + } else { + if (isDebugEnabled) { + log.debug("Allowed because we have permissions for {}", actions); + } + presponse.allowed = true; + + // break the loop as we found the matching permission + break; + } + } + + return presponse; + } + + public Set mapRoles(final User user, final TransportAddress caller) { + return this.configModel.mapSecurityRoles(user, caller); + } + +} diff --git a/src/main/java/org/opensearch/security/resolver/IndexResolverReplacer.java b/src/main/java/org/opensearch/security/resolver/IndexResolverReplacer.java index dff4d4fcba..a907bdbe13 100644 --- a/src/main/java/org/opensearch/security/resolver/IndexResolverReplacer.java +++ b/src/main/java/org/opensearch/security/resolver/IndexResolverReplacer.java @@ -35,12 +35,12 @@ import java.util.List; import java.util.ListIterator; import java.util.Map; +import java.util.Objects; import java.util.Set; import java.util.regex.PatternSyntaxException; import java.util.stream.Collectors; import com.google.common.collect.ImmutableSet; -import org.apache.commons.collections.keyvalue.MultiKey; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.greenrobot.eventbus.Subscribe; @@ -112,7 +112,7 @@ public IndexResolverReplacer(IndexNameExpressionResolver resolver, ClusterServic this.clusterInfoHolder = clusterInfoHolder; } - private static final boolean isAllWithNoRemote(final String... requestedPatterns) { + private static boolean isAllWithNoRemote(final String... requestedPatterns) { final List patterns = requestedPatterns == null ? null : Arrays.asList(requestedPatterns); @@ -131,11 +131,11 @@ private static final boolean isAllWithNoRemote(final String... requestedPatterns return false; } - private static final boolean isLocalAll(String... requestedPatterns) { + private static boolean isLocalAll(String... requestedPatterns) { return isLocalAll(requestedPatterns == null ? null : Arrays.asList(requestedPatterns)); } - private static final boolean isLocalAll(Collection patterns) { + private static boolean isLocalAll(Collection patterns) { if (IndexNameExpressionResolver.isAllIndices(patterns)) { return true; } @@ -158,9 +158,49 @@ private class ResolvedIndicesProvider implements IndicesProvider { private final ImmutableSet.Builder remoteIndices; // set of previously resolved index requests to avoid resolving // the same index more than once while processing bulk requests - private final Set alreadyResolved; + private final Set alreadyResolved; private final String name; + private final class AlreadyResolvedKey { + + private final IndicesOptions indicesOptions; + + private final boolean enableCrossClusterResolution; + + private final String[] original; + + private AlreadyResolvedKey(final IndicesOptions indicesOptions, final boolean enableCrossClusterResolution) { + this(indicesOptions, enableCrossClusterResolution, null); + } + + private AlreadyResolvedKey( + final IndicesOptions indicesOptions, + final boolean enableCrossClusterResolution, + final String[] original + ) { + this.indicesOptions = indicesOptions; + this.enableCrossClusterResolution = enableCrossClusterResolution; + this.original = original; + } + + @Override + public boolean equals(Object o) { + if (this == o) return true; + if (o == null || getClass() != o.getClass()) return false; + AlreadyResolvedKey that = (AlreadyResolvedKey) o; + return enableCrossClusterResolution == that.enableCrossClusterResolution + && Objects.equals(indicesOptions, that.indicesOptions) + && Arrays.equals(original, that.original); + } + + @Override + public int hashCode() { + int result = Objects.hash(indicesOptions, enableCrossClusterResolution); + result = 31 * result + Arrays.hashCode(original); + return result; + } + } + ResolvedIndicesProvider(Object request) { aliases = ImmutableSet.builder(); allIndices = ImmutableSet.builder(); @@ -336,9 +376,13 @@ public String[] provide(String[] original, Object localRequest, boolean supports || localRequest instanceof SearchRequest || localRequest instanceof ResolveIndexAction.Request; // skip the whole thing if we have seen this exact resolveIndexPatterns request - if (alreadyResolved.add( - new MultiKey(indicesOptions, enableCrossClusterResolution, (original != null) ? new MultiKey(original, false) : null) - )) { + final AlreadyResolvedKey alreadyResolvedKey; + if (original != null) { + alreadyResolvedKey = new AlreadyResolvedKey(indicesOptions, enableCrossClusterResolution, original); + } else { + alreadyResolvedKey = new AlreadyResolvedKey(indicesOptions, enableCrossClusterResolution); + } + if (alreadyResolved.add(alreadyResolvedKey)) { resolveIndexPatterns(localRequest.getClass().getSimpleName(), indicesOptions, enableCrossClusterResolution, original); } return IndicesProvider.NOOP; diff --git a/src/main/java/org/opensearch/security/rest/DashboardsInfoAction.java b/src/main/java/org/opensearch/security/rest/DashboardsInfoAction.java index 352d99b57e..ed5b965be2 100644 --- a/src/main/java/org/opensearch/security/rest/DashboardsInfoAction.java +++ b/src/main/java/org/opensearch/security/rest/DashboardsInfoAction.java @@ -65,6 +65,9 @@ public class DashboardsInfoAction extends BaseRestHandler { private final PrivilegesEvaluator evaluator; private final ThreadContext threadContext; + public static final String DEFAULT_PASSWORD_MESSAGE = "Password should be at least 8 characters long and contain at least one " + + "uppercase letter, one lowercase letter, one digit, and one special character."; + public DashboardsInfoAction( final Settings settings, final RestController controller, @@ -103,6 +106,10 @@ public void accept(RestChannel channel) throws Exception { builder.field("multitenancy_enabled", evaluator.multitenancyEnabled()); builder.field("private_tenant_enabled", evaluator.privateTenantEnabled()); builder.field("default_tenant", evaluator.dashboardsDefaultTenant()); + builder.field( + "password_validation_error_message", + client.settings().get(ConfigConstants.SECURITY_RESTAPI_PASSWORD_VALIDATION_ERROR_MESSAGE, DEFAULT_PASSWORD_MESSAGE) + ); builder.endObject(); response = new BytesRestResponse(RestStatus.OK, builder); diff --git a/src/main/java/org/opensearch/security/rest/SecurityWhoAmIAction.java b/src/main/java/org/opensearch/security/rest/SecurityWhoAmIAction.java index 8f20a0b9a2..36af8b4ffa 100644 --- a/src/main/java/org/opensearch/security/rest/SecurityWhoAmIAction.java +++ b/src/main/java/org/opensearch/security/rest/SecurityWhoAmIAction.java @@ -15,6 +15,7 @@ import java.nio.file.Path; import java.util.Collections; import java.util.List; +import java.util.Set; import com.google.common.collect.ImmutableList; import org.apache.logging.log4j.LogManager; @@ -25,6 +26,7 @@ import org.opensearch.core.xcontent.XContentBuilder; import org.opensearch.rest.BaseRestHandler; import org.opensearch.rest.BytesRestResponse; +import org.opensearch.rest.NamedRoute; import org.opensearch.rest.RestChannel; import org.opensearch.rest.RestController; import org.opensearch.rest.RestRequest; @@ -44,7 +46,15 @@ public class SecurityWhoAmIAction extends BaseRestHandler { private static final List routes = addRoutesPrefix( - ImmutableList.of(new Route(GET, "/whoami"), new Route(POST, "/whoami")), + ImmutableList.of( + new Route(GET, "/whoami"), + new Route(POST, "/whoami"), + new NamedRoute.Builder().method(GET) + .path("/whoamiprotected") + .uniqueName("security:whoamiprotected") + .legacyActionNames(Set.of("cluster:admin/opendistro_security/whoamiprotected")) + .build() + ), "/_plugins/_security" ); diff --git a/src/main/java/org/opensearch/security/securityconf/DynamicConfigModelV7.java b/src/main/java/org/opensearch/security/securityconf/DynamicConfigModelV7.java index 3f51524e2c..bce800c811 100644 --- a/src/main/java/org/opensearch/security/securityconf/DynamicConfigModelV7.java +++ b/src/main/java/org/opensearch/security/securityconf/DynamicConfigModelV7.java @@ -29,6 +29,8 @@ import java.net.InetAddress; import java.nio.file.Path; +import java.security.AccessController; +import java.security.PrivilegedAction; import java.util.ArrayList; import java.util.Collections; import java.util.HashSet; @@ -44,6 +46,7 @@ import com.google.common.collect.Multimap; import com.google.common.collect.Multimaps; +import org.opensearch.SpecialPermission; import org.opensearch.common.settings.Settings; import org.opensearch.common.xcontent.XContentType; import org.opensearch.security.auth.AuthDomain; @@ -416,14 +419,11 @@ private void destroyDestroyables(List destroyableComponents) { } private T newInstance(final String clazzOrShortcut, String type, final Settings settings, final Path configPath) { - - String clazz = clazzOrShortcut; - - if (authImplMap.containsKey(clazz + "_" + type)) { - clazz = authImplMap.get(clazz + "_" + type); - } - - return ReflectionHelper.instantiateAAA(clazz, settings, configPath); + final String clazz = authImplMap.computeIfAbsent(clazzOrShortcut + "_" + type, k -> clazzOrShortcut); + return AccessController.doPrivileged((PrivilegedAction) () -> { + SpecialPermission.check(); + return ReflectionHelper.instantiateAAA(clazz, settings, configPath); + }); } private String translateShortcutToClassName(final String clazzOrShortcut, final String type) { diff --git a/src/main/java/org/opensearch/security/transport/SecurityInterceptor.java b/src/main/java/org/opensearch/security/transport/SecurityInterceptor.java index 7bd5024d2c..66f4140d3e 100644 --- a/src/main/java/org/opensearch/security/transport/SecurityInterceptor.java +++ b/src/main/java/org/opensearch/security/transport/SecurityInterceptor.java @@ -43,6 +43,7 @@ import org.opensearch.action.get.GetRequest; import org.opensearch.action.search.SearchAction; import org.opensearch.action.search.SearchRequest; +import org.opensearch.cluster.node.DiscoveryNode; import org.opensearch.cluster.service.ClusterService; import org.opensearch.common.io.stream.StreamInput; import org.opensearch.common.settings.Settings; @@ -131,7 +132,6 @@ public void sendRequestDecorate( TransportRequestOptions options, TransportResponseHandler handler ) { - final Map origHeaders0 = getThreadContext().getHeaders(); final User user0 = getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_USER); final String injectedUserString = getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_USER); @@ -146,6 +146,9 @@ public void sendRequestDecorate( final String origCCSTransientMf = getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_MASKED_FIELD_CCS); final boolean isDebugEnabled = log.isDebugEnabled(); + final DiscoveryNode localNode = OpenSearchSecurityPlugin.getLocalNode(); + boolean isSameNodeRequest = localNode != null && localNode.equals(connection.getNode()); + try (ThreadContext.StoredContext stashedContext = getThreadContext().stashContext()) { final TransportResponseHandler restoringHandler = new RestoringTransportResponseHandler(handler, stashedContext); getThreadContext().putHeader("_opendistro_security_remotecn", cs.getClusterName().value()); @@ -223,7 +226,7 @@ && getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROL getThreadContext().putHeader(headerMap); - ensureCorrectHeaders(remoteAddress0, user0, origin0, injectedUserString, injectedRolesString); + ensureCorrectHeaders(remoteAddress0, user0, origin0, injectedUserString, injectedRolesString, isSameNodeRequest); if (isActionTraceEnabled()) { getThreadContext().putHeader( @@ -249,7 +252,8 @@ private void ensureCorrectHeaders( final User origUser, final String origin, final String injectedUserString, - final String injectedRolesString + final String injectedRolesString, + boolean isSameNodeRequest ) { // keep original address @@ -263,30 +267,49 @@ && getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_ORIGIN_HEADE getThreadContext().putHeader(ConfigConstants.OPENDISTRO_SECURITY_ORIGIN_HEADER, Origin.LOCAL.toString()); } + TransportAddress transportAddress = null; if (remoteAdr != null && remoteAdr instanceof TransportAddress) { - String remoteAddressHeader = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS_HEADER); - if (remoteAddressHeader == null) { - getThreadContext().putHeader( - ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS_HEADER, - Base64Helper.serializeObject(((TransportAddress) remoteAdr).address()) - ); + transportAddress = (TransportAddress) remoteAdr; } } - String userHeader = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER); + // we put headers as transient for same node requests + if (isSameNodeRequest) { + if (transportAddress != null) { + getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS, transportAddress); + } - if (userHeader == null) { if (origUser != null) { - getThreadContext().putHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER, Base64Helper.serializeObject(origUser)); + // if request is going to be handled by same node, we directly put transient value as the thread context is not going to be + // stah. + getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_USER, origUser); } else if (StringUtils.isNotEmpty(injectedRolesString)) { - getThreadContext().putHeader(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_HEADER, injectedRolesString); + getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES, injectedRolesString); } else if (StringUtils.isNotEmpty(injectedUserString)) { - getThreadContext().putHeader(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_USER_HEADER, injectedUserString); + getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_USER, injectedUserString); + } + } else { + if (transportAddress != null) { + getThreadContext().putHeader( + ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS_HEADER, + Base64Helper.serializeObject(transportAddress.address()) + ); } - } + final String userHeader = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER); + if (userHeader == null) { + // put as headers for other requests + if (origUser != null) { + getThreadContext().putHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER, Base64Helper.serializeObject(origUser)); + } else if (StringUtils.isNotEmpty(injectedRolesString)) { + getThreadContext().putHeader(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_HEADER, injectedRolesString); + } else if (StringUtils.isNotEmpty(injectedUserString)) { + getThreadContext().putHeader(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_USER_HEADER, injectedUserString); + } + } + } } private ThreadContext getThreadContext() { diff --git a/src/main/java/org/opensearch/security/transport/SecurityRequestHandler.java b/src/main/java/org/opensearch/security/transport/SecurityRequestHandler.java index d1ad9b02e1..8ea82c9d9d 100644 --- a/src/main/java/org/opensearch/security/transport/SecurityRequestHandler.java +++ b/src/main/java/org/opensearch/security/transport/SecurityRequestHandler.java @@ -95,7 +95,6 @@ protected void messageReceivedDecorate( final TransportChannel transportChannel, Task task ) throws Exception { - String resolvedActionClass = request.getClass().getSimpleName(); if (request instanceof BulkShardRequest) { @@ -142,7 +141,31 @@ protected void messageReceivedDecorate( } // bypass non-netty requests - if (channelType.equals("direct")) { + if (getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_USER) != null + || getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_USER) != null + || getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES) != null + || getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS) != null) { + + final String rolesValidation = getThreadContext().getHeader( + ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_VALIDATION_HEADER + ); + if (!Strings.isNullOrEmpty(rolesValidation)) { + getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_VALIDATION, rolesValidation); + } + + if (isActionTraceEnabled()) { + getThreadContext().putHeader( + "_opendistro_security_trace" + System.currentTimeMillis() + "#" + UUID.randomUUID().toString(), + Thread.currentThread().getName() + + " DIR -> " + + transportChannel.getChannelType() + + " " + + getThreadContext().getHeaders() + ); + } + + putInitialActionClassHeader(initialActionClassValue, resolvedActionClass); + } else { final String userHeader = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER); final String injectedRolesHeader = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_HEADER); final String injectedUserHeader = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_USER_HEADER); @@ -162,15 +185,15 @@ protected void messageReceivedDecorate( ); } - final String originalRemoteAddress = getThreadContext().getHeader( - ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS_HEADER - ); + String originalRemoteAddress = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS_HEADER); if (!Strings.isNullOrEmpty(originalRemoteAddress)) { getThreadContext().putTransient( ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS, new TransportAddress((InetSocketAddress) Base64Helper.deserializeObject(originalRemoteAddress)) ); + } else { + getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS, request.remoteAddress()); } final String rolesValidation = getThreadContext().getHeader( @@ -179,20 +202,9 @@ protected void messageReceivedDecorate( if (!Strings.isNullOrEmpty(rolesValidation)) { getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_VALIDATION, rolesValidation); } + } - if (isActionTraceEnabled()) { - getThreadContext().putHeader( - "_opendistro_security_trace" + System.currentTimeMillis() + "#" + UUID.randomUUID().toString(), - Thread.currentThread().getName() - + " DIR -> " - + transportChannel.getChannelType() - + " " - + getThreadContext().getHeaders() - ); - } - - putInitialActionClassHeader(initialActionClassValue, resolvedActionClass); - + if (channelType.equals("direct")) { super.messageReceivedDecorate(request, handler, transportChannel, task); return; } @@ -272,58 +284,10 @@ protected void messageReceivedDecorate( // network intercluster request or cross search cluster request // CS-SUPPRESS-SINGLE: RegexpSingleline Used to allow/disallow TLS connections to extensions - if (HeaderHelper.isInterClusterRequest(getThreadContext()) + if (!(HeaderHelper.isInterClusterRequest(getThreadContext()) || HeaderHelper.isTrustedClusterRequest(getThreadContext()) - || HeaderHelper.isExtensionRequest(getThreadContext())) { + || HeaderHelper.isExtensionRequest(getThreadContext()))) { // CS-ENFORCE-SINGLE - - final String userHeader = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER); - final String injectedRolesHeader = getThreadContext().getHeader( - ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_HEADER - ); - final String injectedUserHeader = getThreadContext().getHeader( - ConfigConstants.OPENDISTRO_SECURITY_INJECTED_USER_HEADER - ); - - if (Strings.isNullOrEmpty(userHeader)) { - // Keeping role injection with higher priority as plugins under OpenSearch will be using this - // on transport layer - if (!Strings.isNullOrEmpty(injectedRolesHeader)) { - getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES, injectedRolesHeader); - } else if (!Strings.isNullOrEmpty(injectedUserHeader)) { - getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_USER, injectedUserHeader); - } - } else { - getThreadContext().putTransient( - ConfigConstants.OPENDISTRO_SECURITY_USER, - Objects.requireNonNull((User) Base64Helper.deserializeObject(userHeader)) - ); - } - - String originalRemoteAddress = getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS_HEADER); - - if (!Strings.isNullOrEmpty(originalRemoteAddress)) { - getThreadContext().putTransient( - ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS, - new TransportAddress((InetSocketAddress) Base64Helper.deserializeObject(originalRemoteAddress)) - ); - } else { - getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_REMOTE_ADDRESS, request.remoteAddress()); - } - - final String rolesValidation = getThreadContext().getHeader( - ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_VALIDATION_HEADER - ); - if (!Strings.isNullOrEmpty(rolesValidation)) { - getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_INJECTED_ROLES_VALIDATION, rolesValidation); - } - - } else { - // this is a netty request from a non-server node (maybe also be internal: or a shard request) - // and therefore issued by a transport client - - // since OS 2.0 we do not support this any longer because transport client no longer available - final OpenSearchException exception = ExceptionUtils.createTransportClientNoLongerSupportedException(); log.error(exception.toString()); transportChannel.sendResponse(exception); @@ -346,9 +310,8 @@ protected void messageReceivedDecorate( } putInitialActionClassHeader(initialActionClassValue, resolvedActionClass); - - super.messageReceivedDecorate(request, handler, transportChannel, task); } + super.messageReceivedDecorate(request, handler, transportChannel, task); } finally { if (isActionTraceEnabled()) { diff --git a/src/test/java/com/amazon/dlic/auth/http/saml/MockSamlIdpServer.java b/src/test/java/com/amazon/dlic/auth/http/saml/MockSamlIdpServer.java index ef54e3e833..c984b4f670 100644 --- a/src/test/java/com/amazon/dlic/auth/http/saml/MockSamlIdpServer.java +++ b/src/test/java/com/amazon/dlic/auth/http/saml/MockSamlIdpServer.java @@ -23,6 +23,7 @@ import java.net.URISyntaxException; import java.nio.charset.CharsetDecoder; import java.nio.charset.CharsetEncoder; +import java.nio.charset.StandardCharsets; import java.security.GeneralSecurityException; import java.security.KeyStore; import java.security.KeyStoreException; @@ -32,6 +33,8 @@ import java.security.UnrecoverableKeyException; import java.security.cert.CertificateException; import java.security.cert.X509Certificate; +import java.time.Instant; +import java.time.temporal.ChronoUnit; import java.util.Arrays; import java.util.Collections; import java.util.Enumeration; @@ -60,6 +63,7 @@ import javax.xml.transform.stream.StreamResult; import net.shibboleth.utilities.java.support.codec.Base64Support; +import net.shibboleth.utilities.java.support.codec.EncodingException; import net.shibboleth.utilities.java.support.component.ComponentInitializationException; import org.apache.hc.core5.function.Callback; import org.apache.hc.core5.http.ClassicHttpRequest; @@ -82,7 +86,6 @@ import org.apache.hc.core5.http.message.BasicHttpRequest; import org.apache.hc.core5.http.protocol.HttpContext; import org.apache.hc.core5.net.URIBuilder; -import org.joda.time.DateTime; import org.opensaml.core.xml.XMLObject; import org.opensaml.core.xml.XMLObjectBuilderFactory; import org.opensaml.core.xml.config.XMLObjectProviderRegistrySupport; @@ -92,7 +95,6 @@ import org.opensaml.messaging.context.MessageContext; import org.opensaml.messaging.decoder.MessageDecodingException; import org.opensaml.messaging.handler.MessageHandlerException; -import org.opensaml.saml.common.SAMLObject; import org.opensaml.saml.common.SAMLVersion; import org.opensaml.saml.common.messaging.context.SAMLPeerEntityContext; import org.opensaml.saml.common.messaging.context.SAMLProtocolContext; @@ -331,11 +333,11 @@ public String handleSsoGetRequestBase(HttpRequest request) { HTTPRedirectDeflateDecoder decoder = new HTTPRedirectDeflateDecoder(); decoder.setParserPool(XMLObjectProviderRegistrySupport.getParserPool()); - decoder.setHttpServletRequest(httpServletRequest); + decoder.setHttpServletRequestSupplier(() -> httpServletRequest); decoder.initialize(); decoder.decode(); - MessageContext messageContext = decoder.getMessageContext(); + MessageContext messageContext = decoder.getMessageContext(); if (!(messageContext.getMessage() instanceof AuthnRequest)) { throw new RuntimeException("Expected AuthnRequest; received: " + messageContext.getMessage()); @@ -357,7 +359,6 @@ public void handleSloGetRequestURI(String samlRequestURI) { handleSloGetRequestBase(new BasicHttpRequest("GET", samlRequestURI)); } - @SuppressWarnings("unchecked") public void handleSloGetRequestBase(HttpRequest request) { try { @@ -365,11 +366,11 @@ public void handleSloGetRequestBase(HttpRequest request) { HTTPRedirectDeflateDecoder decoder = new HTTPRedirectDeflateDecoder(); decoder.setParserPool(XMLObjectProviderRegistrySupport.getParserPool()); - decoder.setHttpServletRequest(httpServletRequest); + decoder.setHttpServletRequestSupplier(() -> httpServletRequest); decoder.initialize(); decoder.decode(); - MessageContext messageContext = decoder.getMessageContext(); + MessageContext messageContext = decoder.getMessageContext(); if (!(messageContext.getMessage() instanceof LogoutRequest)) { throw new RuntimeException("Expected LogoutRequest; received: " + messageContext.getMessage()); @@ -391,7 +392,7 @@ public void handleSloGetRequestBase(HttpRequest request) { validationParams.setSignatureTrustEngine(buildSignatureTrustEngine(this.spSignatureCertificate)); securityParametersContext.setSignatureValidationParameters(validationParams); - signatureSecurityHandler.setHttpServletRequest(httpServletRequest); + signatureSecurityHandler.setHttpServletRequestSupplier(() -> httpServletRequest); signatureSecurityHandler.initialize(); signatureSecurityHandler.invoke(messageContext); @@ -415,18 +416,18 @@ private String createSamlAuthResponse(AuthnRequest authnRequest) { response.setVersion(SAMLVersion.VERSION_20); response.setStatus(createStatus(StatusCode.SUCCESS)); - response.setIssueInstant(new DateTime()); + response.setIssueInstant(Instant.now()); Assertion assertion = createSamlElement(Assertion.class); assertion.setID(nextId()); - assertion.setIssueInstant(new DateTime()); + assertion.setIssueInstant(Instant.now()); assertion.setIssuer(createIssuer()); AuthnStatement authnStatement = createSamlElement(AuthnStatement.class); assertion.getAuthnStatements().add(authnStatement); - authnStatement.setAuthnInstant(new DateTime()); + authnStatement.setAuthnInstant(Instant.now()); authnStatement.setSessionIndex(nextId()); authnStatement.setAuthnContext(createAuthnCotext()); @@ -440,7 +441,7 @@ private String createSamlAuthResponse(AuthnRequest authnRequest) { .add( createSubjectConfirmation( "urn:oasis:names:tc:SAML:2.0:cm:bearer", - new DateTime().plusMinutes(1), + Instant.now().plus(1, ChronoUnit.MINUTES), authnRequest.getID(), authnRequest.getAssertionConsumerServiceURL() ) @@ -450,7 +451,7 @@ private String createSamlAuthResponse(AuthnRequest authnRequest) { .add( createSubjectConfirmation( "urn:oasis:names:tc:SAML:2.0:cm:bearer", - new DateTime().plusMinutes(1), + Instant.now().plus(1, ChronoUnit.MINUTES), null, defaultAssertionConsumerService ) @@ -460,8 +461,8 @@ private String createSamlAuthResponse(AuthnRequest authnRequest) { Conditions conditions = createSamlElement(Conditions.class); assertion.setConditions(conditions); - conditions.setNotBefore(new DateTime()); - conditions.setNotOnOrAfter(new DateTime().plusMinutes(1)); + conditions.setNotBefore(Instant.now()); + conditions.setNotOnOrAfter(Instant.now().plus(1, ChronoUnit.MINUTES)); if (authenticateUserRoles != null) { AttributeStatement attributeStatement = createSamlElement(AttributeStatement.class); @@ -501,9 +502,9 @@ private String createSamlAuthResponse(AuthnRequest authnRequest) { String marshalledXml = marshallSamlXml(response); - return Base64Support.encode(marshalledXml.getBytes("UTF-8"), Base64Support.UNCHUNKED); + return Base64Support.encode(marshalledXml.getBytes(StandardCharsets.UTF_8), Base64Support.UNCHUNKED); - } catch (MarshallingException | SignatureException | UnsupportedEncodingException | EncryptionException e) { + } catch (MarshallingException | SignatureException | EncryptionException | EncodingException e) { throw new RuntimeException(e); } } @@ -547,7 +548,7 @@ private NameIDFormat createNameIDFormat(String format) { NameIDFormat nameIdFormat = createSamlElement(NameIDFormat.class); - nameIdFormat.setFormat("urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified"); + nameIdFormat.setURI("urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified"); return nameIdFormat; } @@ -567,7 +568,7 @@ private NameID createNameID(String format, String value) { return nameID; } - private SubjectConfirmation createSubjectConfirmation(String method, DateTime notOnOrAfter, String inResponseTo, String recipient) { + private SubjectConfirmation createSubjectConfirmation(String method, Instant notOnOrAfter, String inResponseTo, String recipient) { SubjectConfirmation result = createSamlElement(SubjectConfirmation.class); result.setMethod(method); @@ -591,7 +592,7 @@ private Issuer createIssuer() { private AuthnContext createAuthnCotext() { AuthnContext authnContext = createSamlElement(AuthnContext.class); AuthnContextClassRef authnContextClassRef = createSamlElement(AuthnContextClassRef.class); - authnContextClassRef.setAuthnContextClassRef(AuthnContext.UNSPECIFIED_AUTHN_CTX); + authnContextClassRef.setURI(AuthnContext.UNSPECIFIED_AUTHN_CTX); authnContext.setAuthnContextClassRef(authnContextClassRef); return authnContext; } diff --git a/src/test/java/org/opensearch/security/SystemIntegratorsTests.java b/src/test/java/org/opensearch/security/SystemIntegratorsTests.java index 8d287dd8a3..3c2195b6f5 100644 --- a/src/test/java/org/opensearch/security/SystemIntegratorsTests.java +++ b/src/test/java/org/opensearch/security/SystemIntegratorsTests.java @@ -179,7 +179,7 @@ public void testInjectedUser() throws Exception { Assert.assertTrue(resc.getBody().contains("\"remote_address\":\"8.8.8.8:8\"")); Assert.assertTrue(resc.getBody().contains("\"backend_roles\":[\"role1\",\"role2\"]")); // mapped by username - Assert.assertTrue(resc.getBody().contains("\"roles\":[\"opendistro_security_all_access\"")); + Assert.assertTrue(resc.getBody().contains("\"opendistro_security_all_access\"")); Assert.assertTrue(resc.getBody().contains("\"custom_attribute_names\":[\"key1\",\"key2\"]")); resc = rh.executeGetRequest( diff --git a/src/test/java/org/opensearch/security/filter/RestPathMatchesTest.java b/src/test/java/org/opensearch/security/filter/RestPathMatchesTest.java new file mode 100644 index 0000000000..9a5335bdb7 --- /dev/null +++ b/src/test/java/org/opensearch/security/filter/RestPathMatchesTest.java @@ -0,0 +1,66 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.security.filter; + +import org.junit.Before; +import org.junit.Test; + +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; + +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.Mockito.mock; + +public class RestPathMatchesTest { + Method restPathMatches; + SecurityRestFilter securityRestFilter; + + @Before + public void setUp() throws NoSuchMethodException { + securityRestFilter = mock(SecurityRestFilter.class); + restPathMatches = SecurityRestFilter.class.getDeclaredMethod("restPathMatches", String.class, String.class); + restPathMatches.setAccessible(true); + } + + @Test + public void testExactMatch() throws InvocationTargetException, IllegalAccessException { + String requestPath = "_plugins/security/api/x/y"; + String handlerPath = "_plugins/security/api/x/y"; + assertTrue((Boolean) restPathMatches.invoke(securityRestFilter, requestPath, handlerPath)); + } + + @Test + public void testPartialMatch() throws InvocationTargetException, IllegalAccessException { + String requestPath = "_plugins/security/api/x/y"; + String handlerPath = "_plugins/security/api/x/z"; + assertFalse((Boolean) restPathMatches.invoke(securityRestFilter, requestPath, handlerPath)); + } + + @Test + public void testNamedParamsMatch() throws InvocationTargetException, IllegalAccessException { + String requestPath = "_plugins/security/api/123/y"; + String handlerPath = "_plugins/security/api/{id}/y"; + assertTrue((Boolean) restPathMatches.invoke(securityRestFilter, requestPath, handlerPath)); + } + + @Test + public void testDifferentPathLength() throws InvocationTargetException, IllegalAccessException { + String requestPath = "_plugins/security/api/x/y/z"; + String handlerPath = "_plugins/security/api/x/y"; + assertFalse((Boolean) restPathMatches.invoke(securityRestFilter, requestPath, handlerPath)); + } + + @Test + public void testDifferentPathSegments() throws InvocationTargetException, IllegalAccessException { + String requestPath = "_plugins/security/api/a/b"; + String handlerPath = "_plugins/security/api/x/y"; + assertFalse((Boolean) restPathMatches.invoke(securityRestFilter, requestPath, handlerPath)); + } +} diff --git a/src/test/java/org/opensearch/security/privileges/RestLayerPrivilegesEvaluatorTest.java b/src/test/java/org/opensearch/security/privileges/RestLayerPrivilegesEvaluatorTest.java new file mode 100644 index 0000000000..3c8145c393 --- /dev/null +++ b/src/test/java/org/opensearch/security/privileges/RestLayerPrivilegesEvaluatorTest.java @@ -0,0 +1,163 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.security.privileges; + +import java.util.Collections; +import java.util.Set; +import java.util.concurrent.atomic.AtomicReference; + +import org.apache.logging.log4j.Logger; +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.opensearch.OpenSearchSecurityException; +import org.opensearch.cluster.service.ClusterService; +import org.opensearch.common.settings.Settings; +import org.opensearch.common.util.concurrent.ThreadContext; +import org.opensearch.core.xcontent.NamedXContentRegistry; +import org.opensearch.security.auditlog.AuditLog; +import org.opensearch.security.configuration.ClusterInfoHolder; +import org.opensearch.security.securityconf.ConfigModel; +import org.opensearch.security.securityconf.DynamicConfigModel; +import org.opensearch.security.securityconf.SecurityRoles; +import org.opensearch.security.user.User; +import org.opensearch.threadpool.ThreadPool; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.mockito.Mockito.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +public class RestLayerPrivilegesEvaluatorTest { + + @Mock + private ClusterService clusterService; + @Mock + private ThreadPool threadPool; + @Mock + private AtomicReference namedXContentRegistry; + @Mock + private ConfigModel configModel; + @Mock + private DynamicConfigModel dcm; + @Mock + private PrivilegesEvaluatorResponse presponse; + @Mock + private Logger log; + + private RestLayerPrivilegesEvaluator privilegesEvaluator; + + private static final User TEST_USER = new User("test_user"); + + @Before + public void setUp() throws InstantiationException, IllegalAccessException { + MockitoAnnotations.openMocks(this); + + ThreadContext context = new ThreadContext(Settings.EMPTY); + when(threadPool.getThreadContext()).thenReturn(context); + + privilegesEvaluator = new RestLayerPrivilegesEvaluator( + clusterService, + threadPool, + mock(AuditLog.class), + mock(ClusterInfoHolder.class), + namedXContentRegistry + ); + privilegesEvaluator.onConfigModelChanged(configModel); + privilegesEvaluator.onDynamicConfigModelChanged(dcm); + + when(log.isDebugEnabled()).thenReturn(false); + } + + @Test + public void testEvaluate_Initialized_Success() { + String action = "action"; + SecurityRoles securityRoles = mock(SecurityRoles.class); + when(configModel.getSecurityRoles()).thenReturn(securityRoles); + when(configModel.getSecurityRoles().filter(Collections.emptySet())).thenReturn(securityRoles); + when(securityRoles.impliesClusterPermissionPermission(action)).thenReturn(false); + + PrivilegesEvaluatorResponse response = privilegesEvaluator.evaluate(TEST_USER, Set.of(action)); + + assertNotNull(response); + assertFalse(response.isAllowed()); + assertFalse(response.getMissingPrivileges().isEmpty()); + assertTrue(response.getResolvedSecurityRoles().isEmpty()); + verify(configModel, times(3)).getSecurityRoles(); + } + + @Test(expected = OpenSearchSecurityException.class) + public void testEvaluate_NotInitialized_ExceptionThrown() throws Exception { + String action = "action"; + privilegesEvaluator.evaluate(TEST_USER, Set.of(action)); + } + + @Test + public void testMapRoles_ReturnsMappedRoles() { + Set mappedRoles = Collections.singleton("role1"); + when(configModel.mapSecurityRoles(any(), any())).thenReturn(mappedRoles); + + Set result = privilegesEvaluator.mapRoles(any(), any()); + + assertEquals(mappedRoles, result); + verify(configModel).mapSecurityRoles(any(), any()); + } + + @Test + public void testEvaluate_Successful_NewPermission() { + String action = "hw:greet"; + SecurityRoles securityRoles = mock(SecurityRoles.class); + when(configModel.getSecurityRoles()).thenReturn(securityRoles); + when(configModel.getSecurityRoles().filter(Collections.emptySet())).thenReturn(securityRoles); + when(securityRoles.impliesClusterPermissionPermission(action)).thenReturn(true); + + PrivilegesEvaluatorResponse response = privilegesEvaluator.evaluate(TEST_USER, Set.of(action)); + + assertTrue(response.allowed); + verify(securityRoles).impliesClusterPermissionPermission(any()); + } + + @Test + public void testEvaluate_Successful_LegacyPermission() { + String action = "cluster:admin/opensearch/hw/greet"; + SecurityRoles securityRoles = mock(SecurityRoles.class); + when(configModel.getSecurityRoles()).thenReturn(securityRoles); + when(configModel.getSecurityRoles().filter(Collections.emptySet())).thenReturn(securityRoles); + when(securityRoles.impliesClusterPermissionPermission(action)).thenReturn(true); + + PrivilegesEvaluatorResponse response = privilegesEvaluator.evaluate(TEST_USER, Set.of(action)); + + assertTrue(response.allowed); + verify(securityRoles).impliesClusterPermissionPermission(any()); + } + + @Test + public void testEvaluate_Unsuccessful() { + String action = "action"; + SecurityRoles securityRoles = mock(SecurityRoles.class); + when(configModel.getSecurityRoles()).thenReturn(securityRoles); + when(configModel.getSecurityRoles().filter(Collections.emptySet())).thenReturn(securityRoles); + when(securityRoles.impliesClusterPermissionPermission(action)).thenReturn(false); + + PrivilegesEvaluatorResponse response = privilegesEvaluator.evaluate(TEST_USER, Set.of(action)); + + assertFalse(response.allowed); + verify(securityRoles).impliesClusterPermissionPermission(any()); + } +} diff --git a/src/test/java/org/opensearch/security/transport/SecurityInterceptorTests.java b/src/test/java/org/opensearch/security/transport/SecurityInterceptorTests.java new file mode 100644 index 0000000000..7291050d6e --- /dev/null +++ b/src/test/java/org/opensearch/security/transport/SecurityInterceptorTests.java @@ -0,0 +1,183 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.security.transport; + +// CS-SUPPRESS-SINGLE: RegexpSingleline Extensions manager used for creating a mock +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import org.opensearch.Version; +import org.opensearch.action.search.PitService; +import org.opensearch.cluster.ClusterName; +import org.opensearch.cluster.node.DiscoveryNode; +import org.opensearch.cluster.service.ClusterService; +import org.opensearch.common.settings.Settings; +import org.opensearch.extensions.ExtensionsManager; +import org.opensearch.indices.IndicesService; +import org.opensearch.repositories.RepositoriesService; +import org.opensearch.security.OpenSearchSecurityPlugin; +import org.opensearch.security.auditlog.AuditLog; +import org.opensearch.security.auth.BackendRegistry; +import org.opensearch.security.configuration.ClusterInfoHolder; +import org.opensearch.security.ssl.SslExceptionHandler; +import org.opensearch.security.ssl.transport.PrincipalExtractor; +import org.opensearch.security.ssl.transport.SSLConfig; +import org.opensearch.security.support.Base64Helper; +import org.opensearch.security.support.ConfigConstants; +import org.opensearch.security.user.User; +import org.opensearch.test.OpenSearchTestCase; +import org.opensearch.test.transport.MockTransport; +import org.opensearch.threadpool.ThreadPool; +import org.opensearch.transport.Transport.Connection; +import org.opensearch.transport.TransportInterceptor.AsyncSender; +import org.opensearch.transport.TransportRequest; +import org.opensearch.transport.TransportRequestOptions; +import org.opensearch.transport.TransportResponse; +import org.opensearch.transport.TransportResponseHandler; +import org.opensearch.transport.TransportService; + +import static java.util.Collections.emptySet; +import static org.junit.Assert.assertEquals; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; +// CS-ENFORCE-SINGLE + +public class SecurityInterceptorTests { + + private SecurityInterceptor securityInterceptor; + + @Mock + private BackendRegistry backendRegistry; + + @Mock + private AuditLog auditLog; + + @Mock + private PrincipalExtractor principalExtractor; + + @Mock + private InterClusterRequestEvaluator requestEvalProvider; + + @Mock + private ClusterService clusterService; + + @Mock + private SslExceptionHandler sslExceptionHandler; + + @Mock + private ClusterInfoHolder clusterInfoHolder; + + @Mock + private SSLConfig sslConfig; + + private Settings settings; + + private ThreadPool threadPool; + + @Before + public void setup() { + MockitoAnnotations.openMocks(this); + settings = Settings.builder() + .put("node.name", SecurityInterceptorTests.class.getSimpleName()) + .put("request.headers.default", "1") + .build(); + threadPool = new ThreadPool(settings); + securityInterceptor = new SecurityInterceptor( + settings, + threadPool, + backendRegistry, + auditLog, + principalExtractor, + requestEvalProvider, + clusterService, + sslExceptionHandler, + clusterInfoHolder, + sslConfig + ); + } + + @Test + public void testSendRequestDecorate() { + + ClusterName clusterName = ClusterName.DEFAULT; + when(clusterService.getClusterName()).thenReturn(clusterName); + + MockTransport transport = new MockTransport(); + TransportService transportService = transport.createTransportService( + Settings.EMPTY, + threadPool, + TransportService.NOOP_TRANSPORT_INTERCEPTOR, + boundTransportAddress -> clusterService.state().nodes().get(SecurityInterceptor.class.getSimpleName()), + null, + emptySet() + ); + + // CS-SUPPRESS-SINGLE: RegexpSingleline Extensions manager used for creating a mock + OpenSearchSecurityPlugin.GuiceHolder guiceHolder = new OpenSearchSecurityPlugin.GuiceHolder( + mock(RepositoriesService.class), + transportService, + mock(IndicesService.class), + mock(PitService.class), + mock(ExtensionsManager.class) + ); + // CS-ENFORCE-SINGLE + + User user = new User("John Doe"); + threadPool.getThreadContext().putTransient(ConfigConstants.OPENDISTRO_SECURITY_USER, user); + + AsyncSender sender = mock(AsyncSender.class); + String action = "testAction"; + TransportRequest request = mock(TransportRequest.class); + TransportRequestOptions options = mock(TransportRequestOptions.class); + TransportResponseHandler handler = mock(TransportResponseHandler.class); + + DiscoveryNode localNode = new DiscoveryNode("local-node", OpenSearchTestCase.buildNewFakeTransportAddress(), Version.CURRENT); + Connection connection1 = transportService.getConnection(localNode); + + DiscoveryNode otherNode = new DiscoveryNode("local-node", OpenSearchTestCase.buildNewFakeTransportAddress(), Version.CURRENT); + Connection connection2 = transportService.getConnection(otherNode); + + // setting localNode value explicitly + OpenSearchSecurityPlugin.setLocalNode(localNode); + + // isSameNodeRequest = true + securityInterceptor.sendRequestDecorate(sender, connection1, action, request, options, handler); + // from thread context inside sendRequestDecorate + doAnswer(i -> { + User transientUser = threadPool.getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_USER); + assertEquals(transientUser, user); + return null; + }).when(sender).sendRequest(any(Connection.class), eq(action), eq(request), eq(options), eq(handler)); + + // from original context + User transientUser = threadPool.getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_USER); + assertEquals(transientUser, user); + assertEquals(threadPool.getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER), null); + + // isSameNodeRequest = false + securityInterceptor.sendRequestDecorate(sender, connection2, action, request, options, handler); + // checking thread context inside sendRequestDecorate + doAnswer(i -> { + String serializedUserHeader = threadPool.getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER); + assertEquals(serializedUserHeader, Base64Helper.serializeObject(user)); + return null; + }).when(sender).sendRequest(any(Connection.class), eq(action), eq(request), eq(options), eq(handler)); + + // from original context + User transientUser2 = threadPool.getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_USER); + assertEquals(transientUser2, user); + assertEquals(threadPool.getThreadContext().getHeader(ConfigConstants.OPENDISTRO_SECURITY_USER_HEADER), null); + + } + +} diff --git a/src/test/resources/roles.yml b/src/test/resources/roles.yml index 3f9c8c1158..47371e1abe 100644 --- a/src/test/resources/roles.yml +++ b/src/test/resources/roles.yml @@ -1405,3 +1405,9 @@ sem-role2: - "sem*" allowed_actions: - "indices:admin/index_template/put" + +who_am_i: + reserved: true + hidden: false + description: "Test role to grant access to /whoami endpoint" + cluster_permissions: [ "security:whoamiprotected" ] diff --git a/src/test/resources/roles_mapping.yml b/src/test/resources/roles_mapping.yml index 2233827e51..68ec0f5e48 100644 --- a/src/test/resources/roles_mapping.yml +++ b/src/test/resources/roles_mapping.yml @@ -498,3 +498,8 @@ sem-role2: hidden: false users: - "sem-user2" +who_am_i: + reserved: false + hidden: false + users: + - "nagilum" diff --git a/tools/install_demo_configuration.bat b/tools/install_demo_configuration.bat index 474014536c..68e39267d4 100755 --- a/tools/install_demo_configuration.bat +++ b/tools/install_demo_configuration.bat @@ -315,7 +315,7 @@ echo plugins.security.enable_snapshot_restore_privilege: true >> "%OPENSEARCH_CO echo plugins.security.check_snapshot_restore_write_privileges: true >> "%OPENSEARCH_CONF_FILE%" echo plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"] >> "%OPENSEARCH_CONF_FILE%" echo plugins.security.system_indices.enabled: true >> "%OPENSEARCH_CONF_FILE%" -echo plugins.security.system_indices.indices: [".plugins-ml-model", ".plugins-ml-task", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models"] >> "%OPENSEARCH_CONF_FILE%" +echo plugins.security.system_indices.indices: [".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models"] >> "%OPENSEARCH_CONF_FILE%" :: network.host >nul findstr /b /c:"network.host" "%OPENSEARCH_CONF_FILE%" && ( diff --git a/tools/install_demo_configuration.sh b/tools/install_demo_configuration.sh index 21174e6f63..33dfc4696d 100755 --- a/tools/install_demo_configuration.sh +++ b/tools/install_demo_configuration.sh @@ -383,7 +383,7 @@ echo "plugins.security.enable_snapshot_restore_privilege: true" | $SUDO_CMD tee echo "plugins.security.check_snapshot_restore_write_privileges: true" | $SUDO_CMD tee -a "$OPENSEARCH_CONF_FILE" > /dev/null echo 'plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]' | $SUDO_CMD tee -a "$OPENSEARCH_CONF_FILE" > /dev/null echo 'plugins.security.system_indices.enabled: true' | $SUDO_CMD tee -a "$OPENSEARCH_CONF_FILE" > /dev/null -echo 'plugins.security.system_indices.indices: [".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models"]' | $SUDO_CMD tee -a "$OPENSEARCH_CONF_FILE" > /dev/null +echo 'plugins.security.system_indices.indices: [".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models"]' | $SUDO_CMD tee -a "$OPENSEARCH_CONF_FILE" > /dev/null #network.host if $SUDO_CMD grep --quiet -i "^network.host" "$OPENSEARCH_CONF_FILE"; then