[Backport 2.x] Synchronize major changes from main #3092

peternied · 2023-08-02T23:17:54Z

Description

Manual backports have been failing for a while, specifically the failure on #2926 clued me into the issue, we've been manually resolving these conflicts, but we should be trying to keep everything in sync as much as possible, this set of backports lumped together attempts to handle that.

Issues Resolved

Resolves [BUG] Backports onto 2.x often run into conflicts #3090

Testing

Lots of /gradlew testClasses, CI checks should verify everything else.

Leaving Draft Checklist

Check in on CI status
~~GitHub workflows need to migrate?~~ Not going to worry about these for the moment
Are other backports stuck waiting that should be pulled in?
Clean up commit history so they look like correctly executed backports
~~Note; this change should NOT be squashed merged, I'll manually push so commit history is preserved~~ Actually its been paired down anyways while dealing with other conflicts, just going to squash.

Check List

New functionality includes testing
New functionality has been documented
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

codecov · 2023-08-02T23:33:44Z

Codecov Report

Merging #3092 (2b2f96b) into 2.x (e499a8b) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##                2.x    #3092      +/-   ##
============================================
- Coverage     62.20%   62.18%   -0.03%     
+ Complexity     3314     3309       -5     
============================================
  Files           265      265              
  Lines         19496    19494       -2     
  Branches       3328     3328              
============================================
- Hits          12128    12122       -6     
- Misses         5741     5742       +1     
- Partials       1627     1630       +3

Files Changed	Coverage Δ
...azon/dlic/auth/http/jwt/keybyoidc/JwtVerifier.java	`88.67% <ø> (ø)`
...va/org/opensearch/security/auth/RolesInjector.java	`88.46% <ø> (ø)`
...search/security/transport/SecurityInterceptor.java	`72.07% <ø> (-0.65%)`	⬇️
...dlic/auth/http/saml/AuthTokenProcessorHandler.java	`46.48% <100.00%> (-0.58%)`	⬇️

... and 2 files with indirect coverage changes

build.gradle

…til/ dir Backporting opensearch-project#2823 Signed-off-by: Peter Nied <[email protected]>

This change combines the many updates from the following commits: * 5f62e8a dependabot: bump commons-io:commons-io from 2.11.0 to 2.13.0 (opensearch-project#3074) * 2f69a10 bump com.github.wnameless.json:json-base from 2.4.0 to 2.4.1 (opensearch-project#3062) * c0e50da dependabot: bump org.cryptacular:cryptacular from 1.2.4 to 1.2.5 (opensearch-project#3071) * d3488e8 dependabot: bump kafka_version from 3.5.0 to 3.5.1 (opensearch-project#3041) * ab6778d Update ospackage, checker-qual, zcxvbn and error_prone_annotations, camel-xmlsecurity (opensearch-project#3023) * 0e6608d Bump JSON libs (opensearch-project#2926) * df07bea SAML 4.3.0 addition persmission (opensearch-project#2987) * e5348eb Change maven repo location for compatibility check (opensearch-project#2980) * 4a1ec53 Bump jaxb to 2.3.8 (opensearch-project#2977) * 9599155 Bump guava to 32.1.1-jre (opensearch-project#2976) * 06eed60 dependabot: bump org.glassfish.jaxb:jaxb-runtime from 2.3.4 to 4.0.3 (opensearch-project#2970) * 1113244 Bump eventbus to 3.3.1 (opensearch-project#2965) * 99ff7b3 dependabot: bump org.apache.bcel:bcel from 6.6.0 to 6.7.0 (opensearch-project#2969) * 0794c3f dependabot: bump jakarta.xml.bind:jakarta.xml.bind-api (opensearch-project#2968) * 9e6aab3 dependabot: bump com.google.j2objc:j2objc-annotations from 1.3 to 2.8 (opensearch-project#2963) * 8227f64 dependabot: bump com.sun.istack:istack-commons-runtime (opensearch-project#2960) * 8e044a6 dependabot: bump org.apiguardian:apiguardian-api from 1.0.0 to 1.1.2 (opensearch-project#2964) * 49cbf52 Remove commons-collections 3.2.2 (opensearch-project#2924) * 092e8f5 Bump SAML libs (opensearch-project#2927) * 8ab7cb4 Resolve CVE-2023-2976 by forcing use of Guava 32.0.1 (opensearch-project#2937) * 4eef662 Clean up and bump Apache libs (opensearch-project#2925) * 9a72355 Bump BouncyCastle from jdk15on to jdk15to18 (opensearch-project#2901) * e4f4817 [Enhancement] Parallel test jobs for CI (opensearch-project#2861) * d871af3 Update snappy to 1.1.10.1 and guava to 32.0.1-jre (opensearch-project#2886) * c808692 Format everything (opensearch-project#2866) Signed-off-by: Peter Nied <[email protected]>

Signed-off-by: Andrey Pleskach <[email protected]>

Signed-off-by: Peter Nied <[email protected]>

peternied · 2023-08-03T20:57:08Z

@cwperks @willyborankin Could I get another look?

willyborankin · 2023-08-03T21:03:59Z

@cwperks @willyborankin Could I get another look?

LGTM.

cwperks · 2023-08-03T21:31:35Z

build.gradle

@@ -41,6 +47,8 @@ buildscript {
        maven { url "https://plugins.gradle.org/m2/" }
        maven { url "https://aws.oss.sonatype.org/content/repositories/snapshots" }
        maven { url "https://d1nvenhzbhpy0q.cloudfront.net/snapshots/lucene/" }
+        maven { url "https://build.shibboleth.net/nexus/content/groups/public" }


The shibboleth repositories aren't needed after the reversion, but no harm in having them either.

cwperks · 2023-08-03T21:35:46Z

src/main/java/com/amazon/dlic/auth/http/saml/AuthTokenProcessorHandler.java

@@ -168,9 +168,7 @@ private AuthTokenProcessorAction.Response handleImpl(

        try {

-            SamlResponse samlResponse = new SamlResponse(saml2Settings, null);


Is this change fine even after reverting back to the original version of opensaml?

@RyanL1997

…#3189) ### Description After the upgrade of JJWT from #3092, the JWT Integration Tests in security-dashboards-plugin started failing. See details on: opensearch-project/security-dashboards-plugin#1555 The output from the tests in Github actions was not very revealing, but @RyanL1997 pointed out that when running the test locally after creating a local distro of core + security plugin with 2.x it would output a permissions error: ``` Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "getClassLoader") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:485) ~[?:?] at java.security.AccessController.checkPermission(AccessController.java:1068) ~[?:?] at java.lang.SecurityManager.checkPermission(SecurityManager.java:416) ~[?:?] at java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060) ~[?:?] at java.lang.Thread.getContextClassLoader(Thread.java:1493) ~[?:?] at io.jsonwebtoken.impl.lang.Services$1.getClassLoader(Services.java:37) ~[jjwt-impl-0.11.5.jar:0.11.5] at io.jsonwebtoken.impl.lang.Services.loadFirst(Services.java:100) ~[jjwt-impl-0.11.5.jar:0.11.5] at io.jsonwebtoken.impl.DefaultJwtParserBuilder.build(DefaultJwtParserBuilder.java:191) ~[jjwt-impl-0.11.5.jar:0.11.5] at com.amazon.dlic.auth.http.jwt.HTTPJwtAuthenticator.<init>(HTTPJwtAuthenticator.java:135) ~[opensearch-security-2.10.0.0-SNAPSHOT.jar:2.10.0.0-SNAPSHOT] ... ``` This PR surrounds the call to build the parser with `AccessController.doPrivileged(...)` to ensure the call has the privileges needed to create the parser. This change should also be applied to main. For some reason, the tests are not failing when running on the main branch but are on 2.x. * Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation) Bug fix ### Issues Resolved opensearch-project/security-dashboards-plugin#1555 ### Check List - [ ] New functionality includes testing - [ ] New functionality has been documented - [ ] Commits are signed per the DCO using --signoff By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). Signed-off-by: Craig Perkins <[email protected]>

peternied changed the base branch from main to 2.x August 2, 2023 23:18

peternied force-pushed the backport-3090 branch from 277481a to f12d639 Compare August 2, 2023 23:21

willyborankin reviewed Aug 3, 2023

View reviewed changes

build.gradle Outdated Show resolved Hide resolved

peternied added 2 commits August 3, 2023 15:14

[Backport 2.x] Updates the style of all java files under the */dlic/u…

40f6c86

…til/ dir Backporting opensearch-project#2823 Signed-off-by: Peter Nied <[email protected]>

peternied force-pushed the backport-3090 branch from 895e8a4 to 07eca3a Compare August 3, 2023 15:39

willyborankin and others added 2 commits August 3, 2023 17:51

[Backport 2.x] Clean up and bump Apache libs (opensearch-project#2925)

a363716

Signed-off-by: Andrey Pleskach <[email protected]>

Seperate OpenSAMLpgrade from other changes

b303e16

Signed-off-by: Peter Nied <[email protected]>

peternied force-pushed the backport-3090 branch from 07eca3a to b303e16 Compare August 3, 2023 18:10

Add trailing whitespace

da1249b

Signed-off-by: Peter Nied <[email protected]>

peternied marked this pull request as ready for review August 3, 2023 19:38

peternied requested review from cliu123, DarshitChanpura, davidlago, RyanL1997, stephen-crawford and reta as code owners August 3, 2023 19:38

Walk back to an older version of java-support

2b2f96b

Signed-off-by: Peter Nied <[email protected]>

willyborankin approved these changes Aug 3, 2023

View reviewed changes

peternied merged commit 159698f into opensearch-project:2.x Aug 3, 2023

peternied deleted the backport-3090 branch August 3, 2023 21:31

cwperks reviewed Aug 3, 2023

View reviewed changes

cwperks mentioned this pull request Aug 16, 2023

[2.x] Fix permission issue when creating JWTParser after jjwt upgrade #3189

Merged

3 tasks

cwperks mentioned this pull request Oct 12, 2023

[Backport 2.x] Bump org.apache.zookeeper:zookeeper to 3.9.1, io.netty:netty-tcnative-boringssl-static to 2.0.61.Final and commons-io:commons-io to 2.14.0 #3535

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport 2.x] Synchronize major changes from main #3092

[Backport 2.x] Synchronize major changes from main #3092

peternied commented Aug 2, 2023 •

edited

Loading

codecov bot commented Aug 2, 2023 •

edited

Loading

peternied commented Aug 3, 2023

willyborankin commented Aug 3, 2023

cwperks Aug 3, 2023

cwperks Aug 3, 2023

		@@ -168,9 +168,7 @@ private AuthTokenProcessorAction.Response handleImpl(

		try {

		SamlResponse samlResponse = new SamlResponse(saml2Settings, null);

[Backport 2.x] Synchronize major changes from main #3092

[Backport 2.x] Synchronize major changes from main #3092

Conversation

peternied commented Aug 2, 2023 • edited Loading

Description

Issues Resolved

Testing

Leaving Draft Checklist

Check List

codecov bot commented Aug 2, 2023 • edited Loading

Codecov Report

peternied commented Aug 3, 2023

willyborankin commented Aug 3, 2023

cwperks Aug 3, 2023

Choose a reason for hiding this comment

cwperks Aug 3, 2023

Choose a reason for hiding this comment

peternied commented Aug 2, 2023 •

edited

Loading

codecov bot commented Aug 2, 2023 •

edited

Loading