Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix flaky tests #278

Closed
ylwu-amzn opened this issue Oct 28, 2021 · 15 comments · Fixed by #372
Closed

Fix flaky tests #278

ylwu-amzn opened this issue Oct 28, 2021 · 15 comments · Fixed by #372
Labels
enhancement New feature or request

Comments

@ylwu-amzn
Copy link
Collaborator

Some test case may fail in CI, if you see any test failure, please put it here to track.
@ylwu-amzn ylwu-amzn added the enhancement New feature or request label Oct 28, 2021
@ylwu-amzn
Copy link
Collaborator Author

https://github.com/opensearch-project/anomaly-detection/runs/4038143328?check_suite_focus=true

REPRODUCE WITH: ./gradlew ‘:test’ --tests “org.opensearch.ad.transport.AnomalyDetectorJobTransportActionTests.testStartHistoricalAnalysisForMultiCategoryHCWithUser” -Dtests.seed=D520A92544F25C7F -Dtests.security.manager=false -Dtests.locale=es-UY -Dtests.timezone=America/Metlakatla -Druntime.java=14
org.opensearch.ad.transport.AnomalyDetectorJobTransportActionTests > testStartHistoricalAnalysisForMultiCategoryHCWithUser FAILED
 java.lang.AssertionError
 at __randomizedtesting.SeedInfo.seed([D520A92544F25C7F:4F28F02B8B2BECD7]:0)
 at org.junit.Assert.fail(Assert.java:86)
 at org.junit.Assert.assertTrue(Assert.java:41)
 at org.junit.Assert.assertTrue(Assert.java:52)
 at org.opensearch.ad.transport.AnomalyDetectorJobTransportActionTests.testStartHistoricalAnalysisForMultiCategoryHCWithUser(AnomalyDetectorJobTransportActionTests.java:234)

This error happens sometimes due to memory circuit breaker triggered

@kaituo
Copy link
Collaborator

kaituo commented Oct 28, 2021

Maybe we should retry start detector.

https://github.com/opensearch-project/anomaly-detection/runs/4038562071?check_suite_focus=true

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testDeleteAnomalyDetectorWithRunningAdJob" -Dtests.seed=C35D1E3B844CB432 -Dtests.security.manager=false -Dtests.locale=en-MT -Dtests.timezone=Atlantic/Cape_Verde -Druntime.java=14

org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testDeleteAnomalyDetectorWithRunningAdJob FAILED
org.opensearch.client.ResponseException: method [POST], host [http://[::1]:36207], URI [/_plugins/_anomaly_detection/detectors/fBVPyHwB8Nz3E5X9aZbe/_start], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"status_exception","reason":"Fail to start detector"}],"type":"status_exception","reason":"Fail to start detector"},"status":500}
at __randomizedtesting.SeedInfo.seed([C35D1E3B844CB432:C084E901C92E213A]:0)
at org.opensearch.client.RestClient.convertResponse(RestClient.java:339)
at org.opensearch.client.RestClient.performRequest(RestClient.java:309)
at org.opensearch.client.RestClient.performRequest(RestClient.java:283)
at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:207)
at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:180)
at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:169)
at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testDeleteAnomalyDetectorWithRunningAdJob(AnomalyDetectorRestApiIT.java:722)

65 tests completed, 1 failed, 1 skipped
Tests with failures:

  • org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testDeleteAnomalyDetectorWithRunningAdJob

@kaituo
Copy link
Collaborator

kaituo commented Nov 5, 2021

https://github.com/opensearch-project/anomaly-detection/runs/4112124177?check_suite_focus=true

org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testDeleteAnomalyDetector FAILED
org.opensearch.client.ResponseException: method [DELETE], host [http://[::1]:42677], URI [/_plugins/_anomaly_detection/detectors/Bv2Y7XwBevVJFc-g9ZBy], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"status_exception","reason":"Fail to delete detector"}],"type":"status_exception","reason":"Fail to delete detector"},"status":500}
at __randomizedtesting.SeedInfo.seed([EF4FF755D3A39022:94A0D35D93462922]:0)
at org.opensearch.client.RestClient.convertResponse(RestClient.java:339)
at org.opensearch.client.RestClient.performRequest(RestClient.java:309)
at org.opensearch.client.RestClient.performRequest(RestClient.java:283)
at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:215)
at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:188)
at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:177)
at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testDeleteAnomalyDetector(AnomalyDetectorRestApiIT.java:691)
1> [2021-11-05T10:56:16,689][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testExecuteAnomalyDetectorWithNullDetectorId] before test
1> [2021-11-05T10:56:16,712][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testExecuteAnomalyDetectorWithNullDetectorId] initializing REST clients against [http://[::1]:33635,

@ylwu-amzn
Copy link
Collaborator Author

https://github.com/opensearch-project/anomaly-detection/runs/4159680147?check_suite_focus=true

org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testStartAdJobWithExistingDetector FAILED
    org.opensearch.client.ResponseException: method [POST], host [http://[::1]:39831], URI [/_plugins/_anomaly_detection/detectors/jRheB30BXhX-LeTeJB0U/_start], status line [HTTP/1.1 500 Internal Server Error]
    {"error":{"root_cause":[{"type":"status_exception","reason":"Fail to start detector"}],"type":"status_exception","reason":"Fail to start detector"},"status":500}
        at __randomizedtesting.SeedInfo.seed([9EC5558E86CD0B61:3725F930F61BD72A]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:344)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:215)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:188)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:177)
        at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testStartAdJobWithExistingDetector(AnomalyDetectorRestApiIT.java:863)

@ylwu-amzn
Copy link
Collaborator Author

https://github.com/opensearch-project/anomaly-detection/runs/4159861729?check_suite_focus=true

org.opensearch.ad.e2e.DetectionResultEvalutationIT > testDataset FAILED

    java.lang.AssertionError

        at __randomizedtesting.SeedInfo.seed([6C804271082368E6:7ED8FAE9F5BE9149]:0)
Suite: Test class org.opensearch.ad.e2e.DetectionResultEvalutationIT
        at org.junit.Assert.fail(Assert.java:86)
  2> REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset" -Dtests.seed=6C804271082368E6 -Dtests.security.manager=false -Dtests.locale=ar-LB -Dtests.timezone=America/Bogota -Druntime.java=14
        at org.junit.Assert.assertTrue(Assert.java:41)
  2> java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([6C804271082368E6:7ED8FAE9F5BE9149]:0)
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyTestResults(DetectionResultEvalutationIT.java:104)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyAnomaly(DetectionResultEvalutationIT.java:86)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset(DetectionResultEvalutationIT.java:60)

@ylwu-amzn
Copy link
Collaborator Author

https://github.com/opensearch-project/anomaly-detection/runs/4161894969?check_suite_focus=true

Multi nodes integration testing

Dtests.security.manager=false -Dtests.locale=ro -Dtests.timezone=US/Eastern -Druntime.java=14
org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testAllProfileAnomalyDetector FAILED
    org.opensearch.client.ResponseException: method [GET], host [http://[::1]:40879], URI [/_plugins/_anomaly_detection/detectors/_z-lCH0B511lK4zgmRzd/_profile?_all=true], status line [HTTP/1.1 500 Internal Server Error]
    {"error":{"root_cause":[{"type":"status_exception","reason":"Fail to get detector"}],"type":"status_exception","reason":"Fail to get detector"},"status":500}
        at __randomizedtesting.SeedInfo.seed([EF7802E37F882422:EF95CBE7803911F8]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:344)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:215)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:188)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:177)
        at org.opensearch.ad.AnomalyDetectorRestTestCase.getDetectorProfile(AnomalyDetectorRestTestCase.java:328)
        at org.opensearch.ad.AnomalyDetectorRestTestCase.getDetectorProfile(AnomalyDetectorRestTestCase.java:343)
        at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testAllProfileAnomalyDetector(AnomalyDetectorRestApiIT.java:1122)

@ylwu-amzn
Copy link
Collaborator Author

https://github.com/opensearch-project/anomaly-detection/runs/4181254101?check_suite_focus=true

Multi nodes integration testing

org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testUpdateAnomalyDetectorNameToNew FAILED
    org.opensearch.client.ResponseException: method [PUT], host [http://[::1]:40283], URI [/_plugins/_anomaly_detection/detectors/3ukrEH0BmZaALEA4zs0C?refresh=true], status line [HTTP/1.1 500 Internal Server Error]
    {"error":{"root_cause":[{"type":"status_exception","reason":"Fail to update detector"}],"type":"status_exception","reason":"Fail to update detector"},"status":500}
        at __randomizedtesting.SeedInfo.seed([DF9B6D61B2ABECE7:E3CE6ACF01548DFF]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:344)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:215)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:188)
        at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testUpdateAnomalyDetectorNameToNew(AnomalyDetectorRestApiIT.java:369)

@ylwu-amzn
Copy link
Collaborator Author

https://github.com/opensearch-project/anomaly-detection/runs/4566892159?check_suite_focus=true

Suite: Test class org.opensearch.ad.e2e.DetectionResultEvalutationIT
  2> REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset" -Dtests.seed=5288F4B6704977B0 -Dtests.security.manager=false -Dtests.locale=cs -Dtests.timezone=Atlantic/Madeira -Druntime.java=14
  2> java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([5288F4B6704977B0:40D04C2E8DD48E1F]:0)
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyTestResults(DetectionResultEvalutationIT.java:104)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyAnomaly(DetectionResultEvalutationIT.java:86)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset(DetectionResultEvalutationIT.java:60)
  2> NOTE: leaving temporary files on disk at: /home/runner/work/anomaly-detection/anomaly-detection/build/testrun/integTest/temp/org.opensearch.ad.e2e.DetectionResultEvalutationIT_5288F4B6704977B0-001
  2> NOTE: test params are: codec=Asserting(Lucene87): {}, docValues:{}, maxPointsInLeafNode=1301, maxMBSortInHeap=6.173069679971252, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=cs, timezone=Atlantic/Madeira
  2> NOTE: Linux 5.11.0-1022-azure amd64/Azul Systems, Inc. 14.0.2 (64-bit)/cpus=2,threads=1,free=366458144,total=536870912
  2> NOTE: All tests run in this JVM: [DetectionResultEvalutationIT]
    java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([5288F4B6704977B0:40D04C2E8DD48E1F]:0)
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyTestResults(DetectionResultEvalutationIT.java:104)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyAnomaly(DetectionResultEvalutationIT.java:86)
        at org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset(DetectionResultEvalutationIT.java:60)

@ylwu-amzn ylwu-amzn mentioned this issue Dec 18, 2021
Merged
@ylwu-amzn
Copy link
Collaborator Author

ylwu-amzn commented Dec 18, 2021 •
edited
Loading

Rule violated for class org.opensearch.ad.task.ADTaskManager: lines covered ratio is 0.74, but expected minimum is 0.75
But ./gradlew build run successfully locally.
Seems some test doesn't run in CI.

@amitgalitz amitgalitz mentioned this issue Jan 6, 2022
Closed
@amitgalitz
Copy link
Member

org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testUpdateAnomalyDetectorNameToNew test failed.

https://github.com/opensearch-project/anomaly-detection/runs/4729923577?check_suite_focus=true

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testUpdateAnomalyDetectorNameToNew" -Dtests.seed=7ACE51DE24920D8D -Dtests.security.manager=false -Dtests.locale=zh-TW -Dtests.timezone=Asia/Bangkok -Druntime.java=14
org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testUpdateAnomalyDetectorNameToNew FAILED
    org.opensearch.client.ResponseException: method [PUT], host [http://[::1]:38601], URI [/_plugins/_anomaly_detection/detectors/07pwMH4BrFB264laoiCO?refresh=true], status line [HTTP/1.1 500 Internal Server Error]
    {"error":{"root_cause":[{"type":"status_exception","reason":"Fail to update detector"}],"type":"status_exception","reason":"Fail to update detector"},"status":500}
        at __randomizedtesting.SeedInfo.seed([7ACE51DE24920D8D:469B5670976D6C95]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:344)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:215)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:188)
        at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testUpdateAnomalyDetectorNameToNew(AnomalyDetectorRestApiIT.java:369)

  1> [2022-01-07T00:29:33,182][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testUpdateAnomalyDetectorWithNotExistingIndex] before test
  1> [2022-01-07T00:29:33,184][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testUpdateAnomalyDetectorWithNotExistingIndex] initializing REST clients against [http://[::1]:38601, http://127.0.0.1:34119]
  1> [2022-01-07T00:29:33,480][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testUpdateAnomalyDetectorWithNotExistingIndex] after test
  1> [2022-01-07T00:29:33,488][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testBackwardCompatibilityWithOpenDistro] before test
  1> [2022-01-07T00:29:33,738][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testBackwardCompatibilityWithOpenDistro] after test
  1> [2022-01-07T00:29:33,749][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testValidateAnomalyDetectorOnWrongValidationType] before test
  1> [2022-01-07T00:29:33,864][INFO ][o.o.a.r.AnomalyDetectorRestApiIT] [testValidateAnomalyDetectorOnWrongValidationType] after test

@amitgalitz
Copy link
Member

Multi node integration tests workflow failed for these two methods: org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testUpdateAnomalyDetectorNameToNew and org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset
https://github.com/opensearch-project/anomaly-detection/runs/4808829539?check_suite_focus=true

2> REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testUpdateAnomalyDetectorNameToNew" -Dtests.seed=3778233F6038BBCE -Dtests.security.manager=false -Dtests.locale=ar-BH -Dtests.timezone=Europe/Kaliningrad -Druntime.java=14
 2> org.opensearch.client.ResponseException: method [PUT], host [http://[::1]:35419], URI [/_plugins/_anomaly_detection/detectors/Kl0pVX4BtjLj6fAJ489G?refresh=true], status line [HTTP/1.1 500 Internal Server Error]
   {"error":{"root_cause":[{"type":"status_exception","reason":"Fail to update detector"}],"type":"status_exception","reason":"Fail to update detector"},"status":500}
       at __randomizedtesting.SeedInfo.seed([3778233F6038BBCE:B2D2491D3C7DAD6]:0)
       at org.opensearch.client.RestClient.convertResponse(RestClient.java:344)
       at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
       at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
       at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:215)
       at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:188)
       at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testUpdateAnomalyDetectorNameToNew(AnomalyDetectorRestApiIT.java:369)

2> REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset" -Dtests.seed=3778233F6038BBCE -Dtests.security.manager=false -Dtests.locale=et-EE -Dtests.timezone=Pacific/Marquesas -Druntime.java=14
       at org.junit.Assert.assertTrue(Assert.java:52)
 2> java.lang.AssertionError
       at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyTestResults(DetectionResultEvalutationIT.java:104)
       at __randomizedtesting.SeedInfo.seed([3778233F6038BBCE:25209BA79DA54261]:0)
       at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyAnomaly(DetectionResultEvalutationIT.java:86)
       at org.junit.Assert.fail(Assert.java:86)
       at org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset(DetectionResultEvalutationIT.java:60)
       at org.junit.Assert.assertTrue(Assert.java:41)
 1> [2022-01-13T11:14:02,813][INFO ][o.o.a.e.DetectionResultEvalutationIT] [testDataset] before test
       at org.junit.Assert.assertTrue(Assert.java:52)
 1> [2022-01-13T11:14:02,818][INFO ][o.o.a.e.DetectionResultEvalutationIT] [testDataset] initializing REST clients against [http://[::1]:35419, http://127.0.0.1:43405, http://[::1]:46467, http://127.0.0.1:37057, http://[::1]:42395, http://127.0.0.1:37241]
       at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyTestResults(DetectionResultEvalutationIT.java:104)
 1> [2022-01-13T11:15:12,718][INFO ][o.o.a.e.DetectionResultEvalutationIT] [testDataset] after test
       at org.opensearch.ad.e2e.DetectionResultEvalutationIT.verifyAnomaly(DetectionResultEvalutationIT.java:86)
       at org.opensearch.ad.e2e.DetectionResultEvalutationIT.testDataset(DetectionResultEvalutationIT.java:60)

@ohltyler ohltyler mentioned this issue Jan 31, 2022
Closed
4 tasks
@ohltyler
Copy link
Member

ohltyler commented Feb 2, 2022 •
edited
Loading

4 more seen for 1.3.0.0 (Jenkins jobs ohltyler-integ-test-stable #73, #74):

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsOnPopulatedResultIndex" -Dtests.seed=13B888D6F1EC8EE3 -Dtests.security.manager=false -Dtests.locale=mt -Dtests.timezone=America/Araguaina -Druntime.java=14
  2> org.opensearch.client.WarningFailureException: method [HEAD], host [https://localhost:9200], URI [/.opendistro-anomaly-results], status line [HTTP/1.1 404 Not Found]
    Warnings: [this request accesses system indices: [.opendistro_security], but in a future major version, direct access to system indices will be prevented by default]
        at __randomizedtesting.SeedInfo.seed([13B888D6F1EC8EE3:4A7DC31AC218BA49]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:340)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.test.rest.OpenSearchRestTestCase.indexExists(OpenSearchRestTestCase.java:1041)
        at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsOnPopulatedResultIndex(AnomalyDetectorRestApiIT.java:1763)
  2> Fra 01, 2022 9:24:32 WN org.opensearch.client.RestClient logResponse

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsOnEmptyResultIndex" -Dtests.seed=CAA022085C768AAE -Dtests.security.manager=false -Dtests.locale=und -Dtests.timezone=PRT -Druntime.java=14
  2> org.opensearch.client.WarningFailureException: method [HEAD], host [https://localhost:9200], URI [/.opendistro-anomaly-results], status line [HTTP/1.1 404 Not Found]
    Warnings: [this request accesses system indices: [.opendistro_security], but in a future major version, direct access to system indices will be prevented by default]
        at __randomizedtesting.SeedInfo.seed([CAA022085C768AAE:EC683768645B3E52]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:340)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.test.rest.OpenSearchRestTestCase.indexExists(OpenSearchRestTestCase.java:1041)
        at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsOnEmptyResultIndex(AnomalyDetectorRestApiIT.java:1723)

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsOnNonExistentResultIndex" -Dtests.seed=CAA022085C768AAE -Dtests.security.manager=false -Dtests.locale=und -Dtests.timezone=PRT -Druntime.java=14
  2> org.opensearch.client.WarningFailureException: method [HEAD], host [https://localhost:9200], URI [/.opendistro-anomaly-results], status line [HTTP/1.1 404 Not Found]
    Warnings: [this request accesses system indices: [.opendistro_security], but in a future major version, direct access to system indices will be prevented by default]
        at __randomizedtesting.SeedInfo.seed([CAA022085C768AAE:1EA88BE434FB991]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:340)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.test.rest.OpenSearchRestTestCase.indexExists(OpenSearchRestTestCase.java:1041)
        at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsOnNonExistentResultIndex(AnomalyDetectorRestApiIT.java:1684)

2> org.opensearch.client.ResponseException: method [DELETE], host [https://localhost:9200], URI [/_plugins/_anomaly_detection/detectors/7_nat34BBtrTNMLj9z8s], status line [HTTP/1.1 500 Internal Server Error]
  2> {"error":{"root_cause":[{"type":"status_exception","reason":"Detector is running"}],"type":"status_exception","reason":"Detector is running"},"status":500}
  2> 	at org.opensearch.client.RestClient.convertResponse(RestClient.java:344)
  2> 	at org.opensearch.client.RestClient.performRequest(RestClient.java:314)
  2> 	at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
  2> 	at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:210)
  2> 	at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:183)
  2> 	at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:172)
  2> 	at org.opensearch.ad.AnomalyDetectorRestTestCase.deleteAnomalyDetector(AnomalyDetectorRestTestCase.java:182)
  2> 	at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.lambda$testDeleteAnomalyDetectorWhileRunning$31(AnomalyDetectorRestApiIT.java:1198)

@amitgalitz
Copy link
Member

https://github.com/amitgalitz/anomaly-detection-1/runs/4910110279?check_suite_focus=true

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testStopNonExistingAdJob" -Dtests.seed=CE639A217720A2C9 -Dtests.security.manager=false -Dtests.locale=es-HN -Dtests.timezone=Asia/Baghdad -Druntime.java=14

org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testStopNonExistingAdJob FAILED
    org.opensearch.client.ResponseException: method [POST], host [http://127.0.0.1:46121], URI [/_plugins/_anomaly_detection/detectors/1EWRhH4BF9ShmR5PRVJc/_start], status line [HTTP/1.1 500 Internal Server Error]
    {"error":{"root_cause":[{"type":"status_exception","reason":"Fail to start detector"}],"type":"status_exception","reason":"Fail to start detector"},"status":500}
        at __randomizedtesting.SeedInfo.seed([CE639A217720A2C9:D[30](https://github.com/amitgalitz/anomaly-detection-1/runs/4910110279?check_suite_focus=true#step:7:30)CFCBD0F44FE85]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:344)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:[31](https://github.com/amitgalitz/anomaly-detection-1/runs/4910110279?check_suite_focus=true#step:7:31)4)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:210)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:183)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:172)
        at org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testStopNonExistingAdJob(AnomalyDetectorRestApiIT.java:1004)

@amitgalitz
Copy link
Member

https://github.com/amitgalitz/anomaly-detection-1/runs/4858237954?check_suite_focus=true

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.HistoricalAnalysisRestApiIT.testHistoricalAnalysisForSingleEntityDetector" -Dtests.seed=D339C77D91BFE4D6 -Dtests.security.manager=false -Dtests.locale=ar-SY -Dtests.timezone=Etc/Zulu -Druntime.java=14

org.opensearch.ad.rest.HistoricalAnalysisRestApiIT > testHistoricalAnalysisForSingleEntityDetector FAILED
    org.opensearch.client.ResponseException: method [POST], host [http://127.0.0.1:39829], URI [/_plugins/_anomaly_detection/detectors/pn2vbn4Bjwg-9YWQmNC6/_start], status line [HTTP/1.1 500 Internal Server Error]
    {"error":{"root_cause":[{"type":"status_exception","reason":"Fail to start detector"}],"type":"status_exception","reason":"Fail to start detector"},"status":500}
        at __randomizedtesting.SeedInfo.seed([D339C77D91BFE4D6:E564[30](https://github.com/amitgalitz/anomaly-detection-1/runs/4858237954?check_suite_focus=true#step:7:30)1F51E3FE14]:0)
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:344)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:[31](https://github.com/amitgalitz/anomaly-detection-1/runs/4858237954?check_suite_focus=true#step:7:31)4)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:289)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:215)
        at org.opensearch.ad.TestHelpers.makeRequest(TestHelpers.java:188)
        at org.opensearch.ad.AnomalyDetectorRestTestCase.startAnomalyDetector(AnomalyDetectorRestTestCase.java:157)
        at org.opensearch.ad.HistoricalAnalysisRestTestCase.startHistoricalAnalysis(HistoricalAnalysisRestTestCase.java:220)
        at org.opensearch.ad.rest.HistoricalAnalysisRestApiIT.startHistoricalAnalysis(HistoricalAnalysisRestApiIT.java:115)
        at org.opensearch.ad.rest.HistoricalAnalysisRestApiIT.startHistoricalAnalysis(HistoricalAnalysisRestApiIT.java:106)
        at org.opensearch.ad.rest.HistoricalAnalysisRestApiIT.testHistoricalAnalysisForSingleEntityDetector(HistoricalAnalysisRestApiIT.java:62)

@amitgalitz amitgalitz mentioned this issue Feb 3, 2022
Merged
1 task
@ohltyler ohltyler mentioned this issue Feb 4, 2022
Merged
1 task
@ohltyler ohltyler reopened this Feb 8, 2022
@ohltyler
Copy link
Member

ohltyler commented Feb 8, 2022

Reopening until #369 merged

@ohltyler ohltyler closed this as completed Feb 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use admin client for system index updates in integ tests ohltyler/anomaly-detection-2
4 participants
@kaituo @ylwu-amzn @ohltyler @amitgalitz