Install security plugin from individual artifacts #1307

jmazanec15 · 2023-11-14T05:50:28Z

Description

Create a local cluster using the demo certificates that the security plugin provides. The goal of this change is to run security tests directly instead of through using the docker image. It pulls the individual artifacts from the snapshot repo and tries to install them if security is enabled. To do this, in the build.gradle file, if the option -Dsecurity.enabled=true is passed, we configure the testcluster.integTest to install the security plugin. This can be seen in the configureSecurityPlugin option.

To run cluster:

./gradlew run -Dsecurity.enabled=true

curl "https://admin:admin@localhost:9200/_cluster/health?wait_for_nodes=>=1&wait_for_status=yellow" --insecure

To run integ tests:

./gradlew integTest -Dsecurity.enabled=true

Issues Resolved

#901

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

codecov · 2023-11-14T06:06:18Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (feb7968) 85.10% compared to head (535f070) 85.10%.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #1307   +/-   ##
=========================================
  Coverage     85.10%   85.10%           
  Complexity     1251     1251           
=========================================
  Files           162      162           
  Lines          5101     5101           
  Branches        477      477           
=========================================
  Hits           4341     4341           
  Misses          554      554           
  Partials        206      206

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

heemin32 · 2023-11-14T22:11:25Z

build.gradle

@@ -177,6 +195,8 @@ dependencies {
    testImplementation group: 'org.objenesis', name: 'objenesis', version: '3.2'
    testImplementation group: 'net.bytebuddy', name: 'byte-buddy-agent', version: '1.14.7'
    testFixturesImplementation "org.opensearch:common-utils:${version}"
+
+    zipArchive group: 'org.opensearch.plugin', name:'opensearch-security', version: "${opensearch_build}"


Will this add a dependency on security plugin? In other words, should user install security plugin before using knn?

This will add a dependency for when the user wants to spin up a gradlew cluster using security.

For instance, if gradlew run is executed, it will not require this dependency. Only if ./gradlew run -Dsecurity.enabled=1 is run will it pull it.

I verified this by changing the version to 3.1 (which doesnt exist). When running:

./gradlew run -Dsecurity.enabled=1 Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true FAILURE: Build failed with an exception. * Where: Build file '/Users/jmazane/workspace/Opensearch/DockerRunner/k-NN-1/build.gradle' line: 287 * What went wrong: A problem occurred evaluating root project 'opensearch-knn'. > Could not resolve all files for configuration ':zipArchive'. > Could not find org.opensearch.plugin:opensearch-security:3.1. Searched in the following locations: - https://repo.maven.apache.org/maven2/org/opensearch/plugin/opensearch-security/3.1/opensearch-security-3.1.pom - file:/Users/jmazane/.m2/repository/org/opensearch/plugin/opensearch-security/3.1/opensearch-security-3.1.pom - https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/opensearch-security/3.1/opensearch-security-3.1.pom - https://plugins.gradle.org/m2/org/opensearch/plugin/opensearch-security/3.1/opensearch-security-3.1.pom Required by: project :

but ./gradlew run has no issue.

We can further separate the tasks if we want to provide more separation, but left it like this for development purposes.

heemin32 · 2023-11-14T22:49:44Z

build.gradle

+        systemProperty("username", "admin")
+        systemProperty("password", "admin")
+
+        extraConfigFile("admin-cert.pem", new File("$rootDir/src/test/resources/security/admin-cert.pem"))


Have you explored to use plugins.security.ssl.transport.keystore_filepath instead of pem file? Wondering if plugins.security.ssl.transport.keystore_filepath could end up simpler setup?

I tried a little bit, but was not able to get it to work. I dont see any keys in the https://github.com/opensearch-project/OpenSearch/blob/main/buildSrc/src/main/java/org/opensearch/gradle/testclusters/OpenSearchCluster.java#L581 where it the call is getting hung up. I reached out to security team to see if they have more info on it.

cwperks · 2023-11-16T19:22:13Z

build.gradle

-    systemProperty "https", System.getProperty("https")
-    systemProperty "user", System.getProperty("user")
-    systemProperty "password", System.getProperty("password")
+    systemProperty("tests.opensearch.https", "true")


I believe this should be tests.opensearch.secure

See security BWC tests here which spin up testclusters w/ security: https://github.com/opensearch-project/security/blob/main/.github/actions/run-bwc-suite/action.yaml#L47-L49

WIP commit that tries to create a local cluster using the demo certificates that the security plugin provides. The goal of this change is to run security tests directly instead of through using the docker image. It pulls the individual artifacts from the snapshot repo. Right now, it is able to bring up a cluster. However, the wait for cluster health yellow checks are failing, so the tests are not able to run. Signed-off-by: John Mazanec <[email protected]>

Signed-off-by: John Mazanec <[email protected]>

jmazanec15 · 2023-12-29T18:24:50Z

qa/restart-upgrade/src/test/java/org/opensearch/knn/bwc/ModelIT.java

@@ -139,22 +136,6 @@ public void testKNNModelDefault() throws Exception {
        }
    }

-    // KNN Delete Model test for model in Training State
-    public void testDeleteTrainingModel() throws Exception {


This logic no longer works because with @ryanbogan change, we mark models stuck in training state as failed. So, I deleted it. Also, we should avoid directly accessing the system index in future tests.

Signed-off-by: John Mazanec <[email protected]>

CHANGELOG.md

martin-gaievski · 2023-12-29T19:01:38Z

build.gradle

+            cluster.extraConfigFile(file, local)
+        }
+
+        cluster.setting("plugins.security.ssl.transport.pemcert_filepath", "esnode.pem")


can you please include the ref where the list of plugins and indices can be checked? This may be required for future plugin maintenance

What do you mean by plugins/indices that can be checked? Do you mean where this setting is coming from?

any source that we can use in future to update the list of settings if our tests start failing

I see let me add a comment

@martin-gaievski added

DEVELOPER_GUIDE.md

Signed-off-by: John Mazanec <[email protected]>

This reverts commit 0d6222c. Signed-off-by: John Mazanec <[email protected]>

jmazanec15 · 2024-01-02T22:44:51Z

Windows is failing with flaky test: #1368.

jmazanec15 · 2024-01-02T23:03:43Z

https://github.com/opensearch-project/k-NN/actions/runs/7391155507/job/20107434607?pr=1307 unrelated as well.

opensearch-trigger-bot · 2024-01-03T16:33:37Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-1307-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 8d60054ab1d5a71a323d545060841f7063e7eba6
# Push it to GitHub
git push --set-upstream origin backport/backport-1307-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1307-to-2.x.

…#1307) Changes how security tests are executed. Instead of setting up docker container with security enabled, we now can directly spin up a gradle local cluster with security which we can use to run tests against. To enable this option, we just have to pass `-Dsecurity.enabled=true` as a flag. Along with this, some refactoring was done for the ODFERestTestCase for configuring the client and cleaning up. Signed-off-by: John Mazanec <[email protected]> (cherry picked from commit 8d60054)

…#1307) Changes how security tests are executed. Instead of setting up docker container with security enabled, we now can directly spin up a gradle local cluster with security which we can use to run tests against. To enable this option, we just have to pass `-Dsecurity.enabled=true` as a flag. Along with this, some refactoring was done for the ODFERestTestCase for configuring the client and cleaning up. Signed-off-by: John Mazanec <[email protected]>

heemin32 reviewed Nov 14, 2023

View reviewed changes

jmazanec15 requested a review from heemin32 November 14, 2023 22:41

heemin32 reviewed Nov 14, 2023

View reviewed changes

jmazanec15 requested a review from heemin32 November 14, 2023 23:11

jmazanec15 mentioned this pull request Nov 15, 2023

[BUG] Unable to spin up security cluster with SSL config opensearch-project/OpenSearch#11219

Open

cwperks reviewed Nov 16, 2023

View reviewed changes

jmazanec15 mentioned this pull request Dec 19, 2023

Add option to set user for test cluster to perform initial health checks opensearch-project/OpenSearch#11641

Closed

8 tasks

jmazanec15 force-pushed the issue-901 branch 4 times, most recently from adc2d58 to 9bf3e27 Compare December 29, 2023 01:35

jmazanec15 marked this pull request as ready for review December 29, 2023 17:16

jmazanec15 requested review from navneet1v, VijayanB, vamshin, naveentatikonda, junqiu-lei, martin-gaievski and ryanbogan as code owners December 29, 2023 17:16

jmazanec15 added backport 2.x Infrastructure Changes to infrastructure, testing, CI/CD, pipelines, etc. labels Dec 29, 2023

jmazanec15 force-pushed the issue-901 branch from 6b8a2d1 to 6882d45 Compare December 29, 2023 17:17

jmazanec15 added 3 commits December 29, 2023 09:23

Further improvements to getting security working

7887485

Signed-off-by: John Mazanec <[email protected]>

Clean up and add docs

1a54cf8

Signed-off-by: John Mazanec <[email protected]>

jmazanec15 force-pushed the issue-901 branch from 6882d45 to 1a54cf8 Compare December 29, 2023 17:24

jmazanec15 commented Dec 29, 2023

View reviewed changes

Remove test cases that directly update model index

db931fe

Signed-off-by: John Mazanec <[email protected]>

jmazanec15 force-pushed the issue-901 branch from 81a57c1 to db931fe Compare December 29, 2023 18:43

martin-gaievski reviewed Dec 29, 2023

View reviewed changes

jmazanec15 requested a review from martin-gaievski December 29, 2023 19:23

heemin32 mentioned this pull request Dec 29, 2023

Fix flaky tests #1355

Merged

5 tasks

VijayanB reviewed Dec 29, 2023

View reviewed changes

DEVELOPER_GUIDE.md Outdated Show resolved Hide resolved

Address comments

4e7157e

Signed-off-by: John Mazanec <[email protected]>

jmazanec15 requested a review from VijayanB December 29, 2023 21:44

jmazanec15 added 3 commits January 2, 2024 13:42

Add ensure green to single node test

0d6222c

Signed-off-by: John Mazanec <[email protected]>

Remove unused keys

071095b

Signed-off-by: John Mazanec <[email protected]>

Revert "Add ensure green to single node test"

535f070

This reverts commit 0d6222c. Signed-off-by: John Mazanec <[email protected]>

heemin32 approved these changes Jan 2, 2024

View reviewed changes

martin-gaievski approved these changes Jan 2, 2024

View reviewed changes

VijayanB approved these changes Jan 3, 2024

View reviewed changes

jmazanec15 merged commit 8d60054 into opensearch-project:main Jan 3, 2024
50 checks passed

jmazanec15 mentioned this pull request Jan 3, 2024

[Backport 2.x] Install security plugin from individual artifacts #1369

Merged

1 task

jmazanec15 mentioned this pull request Jan 30, 2024

Fix Secure Cluster github actions for K-NN plugin #1067

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Install security plugin from individual artifacts #1307

Install security plugin from individual artifacts #1307

jmazanec15 commented Nov 14, 2023 •

edited

Loading

codecov bot commented Nov 14, 2023 •

edited

Loading

heemin32 Nov 14, 2023 •

edited

Loading

jmazanec15 Nov 14, 2023

heemin32 Nov 14, 2023

jmazanec15 Nov 14, 2023

cwperks Nov 16, 2023

jmazanec15 Dec 29, 2023

martin-gaievski Dec 29, 2023

jmazanec15 Dec 29, 2023

martin-gaievski Dec 29, 2023

jmazanec15 Dec 29, 2023

jmazanec15 Dec 29, 2023

jmazanec15 commented Jan 2, 2024

jmazanec15 commented Jan 2, 2024

opensearch-trigger-bot bot commented Jan 3, 2024

Install security plugin from individual artifacts #1307

Install security plugin from individual artifacts #1307

Conversation

jmazanec15 commented Nov 14, 2023 • edited Loading

Description

Issues Resolved

Check List

codecov bot commented Nov 14, 2023 • edited Loading

Codecov Report

heemin32 Nov 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmazanec15 commented Jan 2, 2024

jmazanec15 commented Jan 2, 2024

opensearch-trigger-bot bot commented Jan 3, 2024

jmazanec15 commented Nov 14, 2023 •

edited

Loading

codecov bot commented Nov 14, 2023 •

edited

Loading

heemin32 Nov 14, 2023 •

edited

Loading