Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix commons-lang3 package version to fix jarhell issue #1288

Closed
wants to merge 3 commits into from

Conversation

b4sjoo
Copy link
Collaborator

@b4sjoo b4sjoo commented Sep 6, 2023

Description

Follows opensearch-project/security-analytics#535 to fix jarhell issue

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Zhangxunmt
Zhangxunmt previously approved these changes Sep 6, 2023
@@ -189,7 +189,7 @@ integTest {
}

testClusters.integTest {
testDistribution = "ARCHIVE"
testDistribution = "INTEG_TEST"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change to INTEG_TEST?

Copy link
Collaborator Author

@b4sjoo b4sjoo Sep 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If not we will get this error in integ_test:

| Caused by: java.lang.IllegalStateException: jar hell!
| class: com.amazonaws.encryptionsdk.CryptoMaterialsManager
| jar1: /local/home/dev/ml-commons/plugin/build/testclusters/integTest-0/distro/3.0.0-ARCHIVE/lib/aws-encryption-sdk-java-2.4.0.jar
| jar2: /local/home/dev/ml-commons/plugin/build/testclusters/integTest-0/distro/3.0.0-ARCHIVE/plugins/.installing-2105471977921602206/aws-encryption-sdk-java-2.4.0.jar

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm.. I suspect we are going to see problems.
INTEG_TEST does not pull in modules which might work for us for tests but when we have to run it with the distribution(release) it will have modules installed and we will end up with JarHell.
Could you manually try installing the plugin with OpenSearch 2.x locally assembled with this change?
Can you verify we dont hit the problem ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. This fix is introduced by @HenryL27 so I think we need to involve him in the loop. If so do you have any idea on fixing this jarhell issue?

Copy link
Collaborator

@zane-neo zane-neo Sep 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried in my local, with/without changing this, using distribution won't have jar hell issue.

@b4sjoo b4sjoo had a problem deploying to ml-commons-cicd-env September 6, 2023 23:24 — with GitHub Actions Failure
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 6, 2023 23:24 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 6, 2023 23:24 — with GitHub Actions Inactive
@b4sjoo b4sjoo had a problem deploying to ml-commons-cicd-env September 6, 2023 23:24 — with GitHub Actions Failure
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 6, 2023 23:43 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 6, 2023 23:43 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 6, 2023 23:43 — with GitHub Actions Inactive
@b4sjoo b4sjoo had a problem deploying to ml-commons-cicd-env September 6, 2023 23:43 — with GitHub Actions Failure
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:01 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:01 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:01 — with GitHub Actions Inactive
@b4sjoo b4sjoo had a problem deploying to ml-commons-cicd-env September 7, 2023 00:01 — with GitHub Actions Failure
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:16 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:16 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:16 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:16 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:16 — with GitHub Actions Inactive
@b4sjoo b4sjoo temporarily deployed to ml-commons-cicd-env September 7, 2023 00:16 — with GitHub Actions Inactive
@codecov
Copy link

codecov bot commented Sep 7, 2023

Codecov Report

Merging #1288 (6cdab70) into main (de860dd) will decrease coverage by 3.43%.
Report is 64 commits behind head on main.
The diff coverage is 76.31%.

@@             Coverage Diff              @@
##               main    #1288      +/-   ##
============================================
- Coverage     82.31%   78.89%   -3.43%     
- Complexity     1920     2141     +221     
============================================
  Files           149      168      +19     
  Lines          7508     8740    +1232     
  Branches        750      877     +127     
============================================
+ Hits           6180     6895     +715     
- Misses          988     1447     +459     
- Partials        340      398      +58     
Flag Coverage Δ
ml-commons 78.89% <76.31%> (-3.43%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...l/engine/algorithms/ad/AnomalyDetectionLibSVM.java 85.71% <0.00%> (-1.13%) ⬇️
...search/ml/engine/algorithms/clustering/KMeans.java 90.47% <0.00%> (-1.46%) ⬇️
.../ml/engine/algorithms/clustering/RCFSummarize.java 82.02% <0.00%> (-0.94%) ⬇️
...rithms/metrics_correlation/MetricsCorrelation.java 0.00% <0.00%> (-36.08%) ⬇️
...ml/engine/algorithms/rcf/BatchRandomCutForest.java 88.00% <0.00%> (-1.19%) ⬇️
...ine/algorithms/rcf/FixedInTimeRandomCutForest.java 84.61% <0.00%> (-0.83%) ⬇️
...engine/algorithms/regression/LinearRegression.java 90.90% <0.00%> (-0.93%) ⬇️
...gine/algorithms/regression/LogisticRegression.java 79.06% <0.00%> (-0.94%) ⬇️
...search/ml/engine/algorithms/sample/SampleAlgo.java 80.00% <0.00%> (-3.34%) ⬇️
...va/org/opensearch/ml/model/MLModelCacheHelper.java 78.28% <0.00%> (-4.25%) ⬇️
... and 70 more

... and 4 files with indirect coverage changes

@b4sjoo
Copy link
Collaborator Author

b4sjoo commented Sep 7, 2023

Close this due to the core PR has been reverted.

@b4sjoo b4sjoo closed this Sep 7, 2023
@b4sjoo b4sjoo deleted the main_jarhell branch September 8, 2023 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants