Add CI for Windows and MacOS platforms #2190

peternied · 2022-10-24T22:46:00Z

Description

Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. #2184

Switch the gradle commands to be platform agnostic via the gradle/gradle-build-action@v2, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the testSpecialUsernames until we can dive deeper. #2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. #2195

Issues Resolved

Closes Add Windows Support #1474

Check List

New functionality includes testing
~~New functionality has been documented~~
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Peter Nied <[email protected]>

src/test/java/org/opensearch/security/ssl/OpenSSLTest.java

peternied · 2022-10-25T13:33:27Z

Signed-off-by: Peter Nied <[email protected]>

codecov-commenter · 2022-10-25T21:52:53Z

Codecov Report

Merging #2190 (7fe3292) into main (966b3fc) will decrease coverage by 0.14%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #2190      +/-   ##
============================================
- Coverage     61.12%   60.97%   -0.15%     
+ Complexity     3264     3262       -2     
============================================
  Files           259      259              
  Lines         18335    18335              
  Branches       3248     3248              
============================================
- Hits          11207    11180      -27     
- Misses         5537     5565      +28     
+ Partials       1591     1590       -1

Impacted Files	Coverage Δ
...org/opensearch/security/support/SecurityUtils.java	`72.30% <100.00%> (ø)`
...curity/util/ratetracking/HeapBasedRateTracker.java	`48.62% <0.00%> (-22.94%)`	⬇️
...a/org/opensearch/security/tools/SecurityAdmin.java	`36.60% <0.00%> (-0.24%)`	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Tracked with opensearch-project#2193 Signed-off-by: Peter Nied <[email protected]>

opensearch-project#2194 Signed-off-by: Peter Nied <[email protected]>

Signed-off-by: Peter Nied <[email protected]>

opensearch-project#2195 Signed-off-by: Peter Nied <[email protected]>

Signed-off-by: Peter Nied <[email protected]>

peternied · 2022-10-26T13:11:44Z

Signed-off-by: Peter Nied <[email protected]>

peternied · 2022-10-26T14:16:42Z

src/test/java/org/opensearch/security/dlic/dlsfls/DlsTest.java

@@ -366,7 +366,7 @@ public void testDlsWithMinDocCountZeroAggregations() throws Exception {

        // Significant Text Aggregation is not impacted.
        // Non-admin user with setting "min_doc_count=0". Expected to only have access to buckets for dept_manager".
-        String query3 = "{\"aggregations\":{\"significant_termX\":{\"significant_terms\":{\"field\":\"termX.keyword\",\"min_doc_count\":0}}}}";
+        String query3 = "{\"size\":100,\"aggregations\":{\"significant_termX\":{\"significant_terms\":{\"field\":\"termX.keyword\",\"min_doc_count\":0}}}}";


This test inserts 17 documents and then does a query for matches on them checking the 'hits'. This hits have a default limit of 10... so this test was failing all the time because it depends if you got the right mix of 10/17 items in the output. By making this size much larger this should prevent this issue from happening again.

Always be careful with defaults!

src/test/java/org/opensearch/security/test/helper/file/FileHelper.java

src/test/java/org/opensearch/security/ssl/OpenSSLTest.java

Signed-off-by: Peter Nied <[email protected]>

peternied · 2022-10-27T20:24:33Z

@opensearch-project/security This change is ready for review. Note; I assuming this change is approved I'll adjust the branch protection rules for the new matrix of builds

opensearch-trigger-bot · 2022-10-28T16:10:16Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2190-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 a57fd0a745a398b1fb7ba005bda44d0e55f5fc88
# Push it to GitHub
git push --set-upstream origin backport/backport-2190-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2190-to-2.x.

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. #2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. #2195 Signed-off-by: Peter Nied <[email protected]> (cherry picked from commit a57fd0a)

peternied · 2022-10-28T17:05:46Z

Manually backporting in #2205

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. #2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. #2195 Signed-off-by: Peter Nied <[email protected]> (cherry picked from commit a57fd0a)

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <[email protected]> (cherry picked from commit a57fd0a)

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <[email protected]> Signed-off-by: Stephen Crawford <[email protected]>

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <[email protected]> (cherry picked from commit a57fd0a) Signed-off-by: Stephen Crawford <[email protected]>

opensearch-trigger-bot · 2022-12-01T19:13:42Z

The backport to 1.3 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.3 1.3
# Navigate to the new working tree
cd .worktrees/backport-1.3
# Create a new branch
git switch --create backport/backport-2190-to-1.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 a57fd0a745a398b1fb7ba005bda44d0e55f5fc88
# Push it to GitHub
git push --set-upstream origin backport/backport-2190-to-1.3
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.3

Then, create a pull request where the base branch is 1.3 and the compare/head branch is backport/backport-2190-to-1.3.

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <[email protected]>

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <[email protected]> (cherry picked from commit a57fd0a)

peternied added 2 commits October 24, 2022 17:27

Add CI for Windows and MacOS platforms

c3995e7

Signed-off-by: Peter Nied <[email protected]>

Remove OpenSSL tests with platform quirks

3fb3d1a

Signed-off-by: Peter Nied <[email protected]>

stephen-crawford reviewed Oct 25, 2022

View reviewed changes

src/test/java/org/opensearch/security/ssl/OpenSSLTest.java Outdated Show resolved Hide resolved

Support for environment variables in other platforms

2a4ce68

Signed-off-by: Peter Nied <[email protected]>

peternied force-pushed the more-platforms branch from 43fa3f5 to 2a4ce68 Compare October 25, 2022 20:05

peternied added 2 commits October 25, 2022 15:22

Handle dash character for MacOS

a44c85a

Signed-off-by: Peter Nied <[email protected]>

Use lazy lookahead to find the env key name

f876447

Signed-off-by: Peter Nied <[email protected]>

Disabling flaky tests in HeapBasedRateTrackerTest

4b31bca

Tracked with opensearch-project#2193 Signed-off-by: Peter Nied <[email protected]>

peternied force-pushed the more-platforms branch from 2d66168 to 4b31bca Compare October 25, 2022 23:35

peternied added 5 commits October 25, 2022 18:41

Disable testSpecialUsernames broken on Windows

d753335

opensearch-project#2194 Signed-off-by: Peter Nied <[email protected]>

Enable retries on integration tests

e704db9

Signed-off-by: Peter Nied <[email protected]>

Disable OpenSSL on Windows and Mac

d9aa6cf

opensearch-project#2195 Signed-off-by: Peter Nied <[email protected]>

Remove env variable for OpenSSL setup

d2a8952

Signed-off-by: Peter Nied <[email protected]>

Fix checkstyle issue

49016cd

Signed-off-by: Peter Nied <[email protected]>

peternied added 2 commits October 26, 2022 08:44

Move validation steps in separate jobs

c937165

Signed-off-by: Peter Nied <[email protected]>

Protect against random failures due to document count

df79cef

Signed-off-by: Peter Nied <[email protected]>

peternied force-pushed the more-platforms branch from cee4700 to df79cef Compare October 26, 2022 13:45

Ensure integrationTest is not run in primary workflow

dbe4669

Signed-off-by: Peter Nied <[email protected]>

peternied commented Oct 26, 2022

View reviewed changes

peternied added 4 commits October 26, 2022 14:49

Verify that removing the clean step unblocks the tests

5c7123e

Signed-off-by: Peter Nied <[email protected]>

Run all impacted tests only

97d1f77

Signed-off-by: Peter Nied <[email protected]>

Merge remote-tracking branch 'origin/main' into more-platforms

5a7154f

Signed-off-by: Peter Nied <[email protected]>

Verify SSL tests are in order

70a4cf3

Signed-off-by: Peter Nied <[email protected]>

peternied mentioned this pull request Oct 26, 2022

Add Windows Support opensearch-project/security-dashboards-plugin#845

Closed

3 tasks

peternied added 3 commits October 26, 2022 16:28

Disable OpenSSL for only Windows

e167fd7

Signed-off-by: Peter Nied <[email protected]>

Fix build break

221ece1

Signed-off-by: Peter Nied <[email protected]>

Clean up for review readiness

5f8211c

Signed-off-by: Peter Nied <[email protected]>

peternied marked this pull request as ready for review October 26, 2022 22:06

peternied requested a review from a team October 26, 2022 22:06

Remove testEnsureOpenSSLAvailabilit that wass unreliable

7fe3292

Signed-off-by: Peter Nied <[email protected]>

peternied added backport 2.x backport to 2.x branch v2.4.0 'Issues and PRs related to version v2.4.0' labels Oct 27, 2022

davidlago approved these changes Oct 28, 2022

View reviewed changes

peternied merged commit a57fd0a into opensearch-project:main Oct 28, 2022

peternied deleted the more-platforms branch October 28, 2022 16:40

peternied mentioned this pull request Oct 31, 2022

[Backport 2.x] Add CI for Windows and MacOS platforms (#2190) #2205

Merged

3 tasks

cwperks mentioned this pull request Nov 3, 2022

2.4 Release Notes for Security Plugin #2231

Merged

3 tasks

stephen-crawford added the backport 1.3 backport to 1.3 branch label Dec 1, 2022

stephen-crawford mentioned this pull request Dec 1, 2022

[BACKPORT] Backport Windows CI #2289

Closed

3 tasks

cwperks mentioned this pull request Dec 1, 2022

Backport Windows support to 1.x branch #2281

Closed

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CI for Windows and MacOS platforms #2190

Add CI for Windows and MacOS platforms #2190

peternied commented Oct 24, 2022 •

edited

Loading

peternied commented Oct 25, 2022 •

edited

Loading

codecov-commenter commented Oct 25, 2022 •

edited

Loading

peternied commented Oct 26, 2022 •

edited

Loading

peternied Oct 26, 2022

peternied commented Oct 27, 2022

opensearch-trigger-bot bot commented Oct 28, 2022

peternied commented Oct 28, 2022

opensearch-trigger-bot bot commented Dec 1, 2022

Add CI for Windows and MacOS platforms #2190

Add CI for Windows and MacOS platforms #2190

Conversation

peternied commented Oct 24, 2022 • edited Loading

Description

Issues Resolved

Check List

peternied commented Oct 25, 2022 • edited Loading

MacOS

Windows

codecov-commenter commented Oct 25, 2022 • edited Loading

Codecov Report

peternied commented Oct 26, 2022 • edited Loading

Latest failures to investigate

JDK11

Linux

Windows

MacOS

JDK17

Linux

Windows

MacOS

peternied Oct 26, 2022

Choose a reason for hiding this comment

peternied commented Oct 27, 2022

opensearch-trigger-bot bot commented Oct 28, 2022

peternied commented Oct 28, 2022

opensearch-trigger-bot bot commented Dec 1, 2022

peternied commented Oct 24, 2022 •

edited

Loading

peternied commented Oct 25, 2022 •

edited

Loading

codecov-commenter commented Oct 25, 2022 •

edited

Loading

peternied commented Oct 26, 2022 •

edited

Loading