Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] integrationTest should have retries of at least 1 enabled #2184

Closed
Tracked by #2264
peternied opened this issue Oct 20, 2022 · 6 comments
Closed
Tracked by #2264

[BUG] integrationTest should have retries of at least 1 enabled #2184

peternied opened this issue Oct 20, 2022 · 6 comments
Labels
bug Something isn't working sprint backlog triaged Issues labeled as 'Triaged' have been reviewed and are deemed actionable.

Comments

@peternied
Copy link
Member

What is the bug?
Random failures of

Tests under ./src/integrationTest/java/org/opensearch/security/** should be called with retries, of at least 1 to prevent blocking pull requests.

@peternied peternied added bug Something isn't working untriaged Require the attention of the repository maintainers and may need to be prioritized labels Oct 20, 2022
@peternied
Copy link
Member Author

@lukasz-soszynski-eliatra Could you look into this?

@lukasz-soszynski-eliatra
Copy link
Contributor

I hope that I found the root cause of the problem. The build output contains the following output:

2022-10-20 17:47:18 opensearch[data_0][transport_worker][T#1] WARN  suppressed:125 - path: /restored_write_song_index/_count, params: {ignore_unavailable=false, expand_wildcards=open, allow_no_indices=true, ignore_throttled=true, index=restored_write_song_index}
org.opensearch.action.search.SearchPhaseExecutionException: all shards failed
	at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:661) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:371) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:696) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:469) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:293) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:104) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:74) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:753) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.TransportService$6.handleException(TransportService.java:759) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.security.transport.SecurityInterceptor$RestoringTransportResponseHandler.handleException(SecurityInterceptor.java:312) ~[main/:?]
	at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1379) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.InboundHandler.lambda$handleException$3(InboundHandler.java:420) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:341) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.InboundHandler.handleException(InboundHandler.java:418) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:410) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.InboundHandler.messageReceived(InboundHandler.java:158) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:114) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:769) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:175) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:150) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:115) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:94) [transport-netty4-client-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) [netty-handler-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1373) [netty-handler-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236) [netty-handler-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) [netty-handler-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) [netty-codec-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) [netty-codec-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) [netty-codec-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:623) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:586) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.79.Final.jar:4.1.79.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.79.Final.jar:4.1.79.Final]
	at java.lang.Thread.run(Thread.java:829) [?:?]

This means that an exception occurred when request /restored_write_song_index/_count was executed because all shards failed. This is expected at some stage during the test. The exception is related to the following line https://github.com/opensearch-project/security/blob/main/src/integrationTest/java/org/opensearch/security/SearchOperationTest.java#L1401

Awaitility.await().until(() -> restHighLevelClient.count(countRequest, DEFAULT).getCount() == 2);

Unfortunately, if an exception occurs then waiting is interrupted and the test fails. Therefore I hope that the following commit corrects the problem
f7d2726

@stephen-crawford
Copy link
Contributor

[TRIAGE] Already has a PR related, awaiting review.

@stephen-crawford stephen-crawford added triaged Issues labeled as 'Triaged' have been reviewed and are deemed actionable. sprint backlog and removed untriaged Require the attention of the repository maintainers and may need to be prioritized labels Oct 24, 2022
peternied added a commit that referenced this issue Oct 28, 2022
Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  #2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  #2195

Signed-off-by: Peter Nied <[email protected]>
peternied added a commit that referenced this issue Oct 28, 2022
Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  #2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  #2195

Signed-off-by: Peter Nied <[email protected]>
(cherry picked from commit a57fd0a)
peternied added a commit that referenced this issue Oct 28, 2022
Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  #2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  #2195

Signed-off-by: Peter Nied <[email protected]>
(cherry picked from commit a57fd0a)
peternied added a commit that referenced this issue Oct 31, 2022
Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  #2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  #2195

Signed-off-by: Peter Nied <[email protected]>
(cherry picked from commit a57fd0a)
stephen-crawford pushed a commit to stephen-crawford/security that referenced this issue Nov 10, 2022
…nsearch-project#2205)

Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  opensearch-project#2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  opensearch-project#2195

Signed-off-by: Peter Nied <[email protected]>
(cherry picked from commit a57fd0a)
stephen-crawford pushed a commit to stephen-crawford/security that referenced this issue Nov 10, 2022
Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  opensearch-project#2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  opensearch-project#2195

Signed-off-by: Peter Nied <[email protected]>
Signed-off-by: Stephen Crawford <[email protected]>
stephen-crawford pushed a commit to stephen-crawford/security that referenced this issue Nov 10, 2022
…nsearch-project#2205)

Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  opensearch-project#2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  opensearch-project#2195

Signed-off-by: Peter Nied <[email protected]>
(cherry picked from commit a57fd0a)
Signed-off-by: Stephen Crawford <[email protected]>
stephen-crawford pushed a commit to stephen-crawford/security that referenced this issue Nov 10, 2022
…nsearch-project#2205)

Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  opensearch-project#2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  opensearch-project#2195

Signed-off-by: Peter Nied <[email protected]>
(cherry picked from commit a57fd0a)
Signed-off-by: Stephen Crawford <[email protected]>
@DarshitChanpura
Copy link
Member

Looks like the retries were added. Can we close this issue @peternied ?

@stephen-crawford
Copy link
Contributor

@DarshitChanpura I think it is fine to close this since the associated PR was merged.

peternied added a commit to peternied/security that referenced this issue Dec 1, 2022
Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  opensearch-project#2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  opensearch-project#2195

Signed-off-by: Peter Nied <[email protected]>
stephen-crawford pushed a commit to stephen-crawford/security that referenced this issue Dec 2, 2022
Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  opensearch-project#2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  opensearch-project#2195

Signed-off-by: Peter Nied <[email protected]>
wuychn pushed a commit to ochprince/security that referenced this issue Mar 16, 2023
…nsearch-project#2205)

Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  opensearch-project#2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  opensearch-project#2195

Signed-off-by: Peter Nied <[email protected]>
(cherry picked from commit a57fd0a)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working sprint backlog triaged Issues labeled as 'Triaged' have been reviewed and are deemed actionable.
Projects
None yet
Development

No branches or pull requests

4 participants