Failing test: Jest Tests.x-pack/plugins/index_lifecycle_management/jest/client_integration/edit_policy/form_validation - <EditPolicy /> timing validation should validate that min_age is equal or greater than previous phase min_age #115307

kibanamachine · 2021-10-17T07:35:04Z

A test failed on a tracked branch

Error: thrown: "Exceeded timeout of 5000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."
    at /opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/form_validation/timing.test.ts:87:3
    at _dispatchDescribe (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-circus/build/index.js:67:26)
    at describe (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-circus/build/index.js:30:5)
    at Object.<anonymous> (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/form_validation/timing.test.ts:15:1)
    at Runtime._execModule (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-runtime/build/index.js:1299:24)
    at Runtime._loadModule (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-runtime/build/index.js:898:12)
    at Runtime.requireModule (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-runtime/build/index.js:746:10)
    at jestAdapter (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:106:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at runTestInternal (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-runner/build/runTest.js:380:16)
    at runTest (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-runner/build/runTest.js:472:34)
    at Object.worker (/opt/local-ssd/buildkite/builds/kb-c2-16-a8225099fc8e1e2a/elastic/kibana-hourly/kibana/node_modules/jest-runner/build/testWorker.js:133:12)

First failure: CI Build - 7.x

The text was updated successfully, but these errors were encountered:

elasticmachine · 2021-10-18T09:14:10Z

Pinging @elastic/kibana-stack-management (Team:Stack Management)

kibanamachine · 2021-10-28T08:43:58Z

New failure: CI Build - master

spalger · 2021-10-28T09:09:06Z

7 failures in the last 7 days

master/8.1: c760db6

kibanamachine · 2021-10-30T08:37:03Z

New failure: CI Build - 8.0

kibanamachine · 2021-11-12T00:39:31Z

New failure: CI Build - 7.16

kibanamachine · 2022-08-17T21:29:15Z

New failure: CI Build - main

spalger · 2022-08-17T22:33:52Z

@sebelga We're seeing similar failures in #139062, it's possible this isn't actually the fault of this specific test. I'm wondering if

kibana/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/helpers/actions/toggle_phase_action.ts

Lines 26 to 32 in 0d77708

    
           await act(async () => { 
        
             if (action === 'disable') { 
        
               button.simulate('click'); 
        
             } else { 
        
               button.find('input').simulate('change'); 
        
             } 
        
           });

might be part of the problem. Is there a chance that act() is continuing to call the callback after the test finishes? It has no way of knowing when the test fails and the logs from these tests are full of Cannot read properties of null (reading 'createEvent') errors which seem to imply events are being simulated after a test file completes.

Logs: https://storage.googleapis.com/ci-artifacts.kibana.dev/logs/0182ad8d-33f8-42af-bc3b-18d5dc5aa35a.logs.html

kibanamachine · 2022-08-23T20:15:18Z

New failure: CI Build - main

sebelga · 2022-08-24T13:56:14Z

It seems related to a jest timeout issue. Is it possible that it times out but pending tasks (clicking on buttons) are still scheduled causing those Cannot read properties of null (reading 'createEvent')?

The fact that it "suddenly" started to fail is strange. Did we make any change to jest?

Looking at this issue it seems that changes to jest could cause a different behaviour (slower execution in tests?).

In those tests we make heavy use of jest.runAllTimers(). This comment refers to using jest.useFakeTimers( 'legacy' ), not sure if we should start doing that. 🤔

Maybe a first solution would be to increase the timeout from 5000 to 10000?

sebelga · 2022-08-24T13:58:54Z

When running locally sometimes I get

# 10.392 seconds with warning "Jest did not exit one second..."

Test Suites: 1 passed, 1 total
Tests:       21 passed, 21 total
Snapshots:   0 total
Time:        10.392 s, estimated 11 s
Ran all test suites matching /x-pack\/plugins\/index_lifecycle_management\/__jest__\/client_integration\/edit_policy\/form_validation\/timing.test.ts/i.
Jest did not exit one second after the test run has completed.

and other times

# 9.832 seconds without the warning

Test Suites: 1 passed, 1 total
Tests:       21 passed, 21 total
Snapshots:   0 total
Time:        9.832 s, estimated 11 s
Ran all test suites matching /x-pack\/plugins\/index_lifecycle_management\/__jest__\/client_integration\/edit_policy\/form_validation\/timing.test.ts/i.
✨  Done in 11.46s.

spalger · 2022-08-24T23:50:19Z

Is it possible that it times out but pending tasks (clicking on buttons) are still scheduled causing those Cannot read properties of null (reading 'createEvent')?

That's definitely the idea, .simulate() seems to be getting called outside of a test, but I don't see any failures or timeouts before this error so it doesn't seem likely that timeout is to blame IMO.

In those tests we make heavy use of jest.runAllTimers(). jestjs/jest#11607 (comment) refers to using jest.useFakeTimers( 'legacy' ), not sure if we should start doing that. 🤔

I have a feeling that's not going to help anything... I wonder if we can create a safe version of act that won't call it's callback if the test which registered the hook has exited. Experimenting with this in #139444

spalger · 2022-08-25T20:57:30Z

Welp, my experiment in #139444 failed. It took thousands of test executions to trigger it again but the test eventually failed and my changes didn't do anything to help identify where it's coming from. https://buildkite.com/elastic/kibana-pull-request/builds/67493#0182d61a-dfc6-4543-84e7-3b75504c355b/4838-5380

sebelga · 2022-08-26T12:22:50Z

This seems to be a known issue when running jest on CI where the machine could have low CPU available jestjs/jest#12670

It seems that increasing the timeout for the test is the way to go (jestjs/jest#12670 (comment), jestjs/jest#12670 (comment))

Someone patched the "react-testing-library" (https://github.com/Zylphrex/react-testing-library-error-repro) to better locate the failing test. Maybe we could try something similar?

spalger · 2022-08-30T20:15:46Z

Trying out the patch idea now https://buildkite.com/elastic/kibana-pull-request/builds/68479

spalger · 2022-08-31T13:52:10Z

Alright, I ran these tests 5000 times and it never triggered the failure condition. I can try again, but I also think that it would be great if we could track some tech-debt to deal with the rest of the errors which are being logged about constantly.

Stuff like:

    console.error
      Warning: Functions are not valid as a React child. This may happen if you return a Component instead of <Component /> from render. Or maybe you meant to call this function rather than return it.
          at div
          at /var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/node_modules/@emotion/react/dist/emotion-element-ae8cc4ba.cjs.dev.js:63:23
          at EuiTextColor (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/node_modules/@elastic/eui/test-env/components/text/text_color.js:42:23)
          at EuiText (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/node_modules/@elastic/eui/test-env/components/text/text.js:35:24)
          at /var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/node_modules/@emotion/react/dist/emotion-element-ae8cc4ba.cjs.dev.js:63:23
          at div
          ...this stack trace is very long...

    console.error
      Warning: An update to AddPolicyToTemplateConfirmModal inside a test was not wrapped in act(...).
      
      When testing, code that causes React state updates should be wrapped into act(...):
      
      act(() => {
        /* fire events that update state */
      });
      /* assert on the output */
      
      This ensures that you're testing the behavior the user would see in the browser. Learn more at https://reactjs.org/link/wrap-tests-with-act
          at AddPolicyToTemplateConfirmModal (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/x-pack/plugins/index_lifecycle_management/public/application/sections/policy_list/components/add_policy_to_template_confirm_modal.tsx:39:3)
          at ListActionHandler (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/x-pack/plugins/index_lifecycle_management/public/application/sections/policy_list/components/list_action_handler.tsx:17:69)
          at PolicyList (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/x-pack/plugins/index_lifecycle_management/public/application/sections/policy_list/policy_list.tsx:24:62)
          at PolicyListContextProvider (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/x-pack/plugins/index_lifecycle_management/public/application/sections/policy_list/policy_list_context.tsx:26:45)
          at Provider (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/src/plugins/kibana_react/public/context/context.tsx:49:15)
          at TestComponent (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/x-pack/plugins/index_lifecycle_management/__jest__/policy_table.test.tsx:116:26)
          at WrapperComponent (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/node_modules/@wojtekmaj/enzyme-adapter-utils/src/createMountWrapper.jsx:46:26)
      102 |           ? deserializer(serializedResponseData)
      103 |           : serializedResponseData;
    > 104 |         setData(responseData);
          |         ^
      105 |       }
      106 |       // There can be situations in which a component that consumes this hook gets unmounted when
      107 |       // the request returns an error. So before changing the isLoading state, check if the component

    console.warn
      Deprecation warning: value provided is not in a recognized RFC2822 or ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions. Non RFC2822/ISO date formats are discouraged. Please refer to http://momentjs.com/guides/#/warnings/js-date/ for more info.
      Arguments: 
      [0] _isAMomentObject: true, _isUTC: false, _useUTC: false, _l: undefined, _i: 1661892751629, _f: undefined, _strict: undefined, _locale: [object Object]
      Error: 
          at Function.createFromInputFallback (/var/lib/buildkite-agent/builds/kb-n2-4-spot-1eb5074c59143e53/elastic/kibana-pull-request/kibana/node_modules/moment/moment.js:324:25)
          ...another very long stack trace...

spalger · 2022-08-31T19:41:48Z

Ran another 5000 times, still no errors... maybe this patch is somehow fixing the problem, or masking it... https://github.com/elastic/kibana/pull/139444/files#diff-ac1bbfd235d20b9165116ea2bede56cc73ffc12ad48bcd190d492b92c2014964

spalger · 2022-09-01T12:10:50Z

Finally got a failure with the patched version: https://buildkite.com/elastic/kibana-pull-request/builds/68843#0182f576-0efc-44cd-a314-cd75d6ac0f7f/726-842

I don't see anything that we can do to make this work...

sebelga · 2022-09-05T10:47:35Z

I also think that it would be great if we could track some tech-debt to deal with the rest of the errors which are being logged about constantly.

You are right, we should clean up and wrap the state change inside act() calls. cc @elastic/platform-deployment-management as those tests fall under their team.

I don't see anything that we can do to make this work...

I don't either. I don't even see what error is thrown. Should we increase the timeout manually in those component integration tests that run slower than unit tests?

spalger · 2022-09-06T16:59:00Z

Should we increase the timeout manually in those component integration tests that run slower than unit tests?

I'd prefer that they are moved to integration_tests directories and run as integration tests instead of unit tests, but if you want to just increase the timeout I can turn away and pretend I didn't see anything 😅

sebelga · 2022-09-08T07:13:17Z

I'll leave that decision to the @elastic/platform-deployment-management team 😄

kibanamachine · 2022-09-20T16:24:02Z

New failure: CI Build - main

jbudz · 2022-09-20T16:29:17Z

Skipped

main: e468632

kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 17, 2021

botelastic bot added the needs-team Issues missing a team label label Oct 17, 2021

azasypkin added the Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more label Oct 18, 2021

botelastic bot removed the needs-team Issues missing a team label label Oct 18, 2021

cjcenizal added the Feature:ILM label Oct 18, 2021

spalger added a commit that referenced this issue Oct 28, 2021

skip flaky suite (#115307)

c760db6

spalger added blocker v8.1.0 labels Oct 28, 2021

academo pushed a commit to academo/kibana that referenced this issue Oct 28, 2021

skip flaky suite (elastic#115307)

6fde826

spalger added the skipped-test label Nov 16, 2021

sebelga mentioned this issue Dec 9, 2021

[ILM] Unskip timing validation component integration test #120967

Merged

sebelga closed this as completed in #120967 Dec 14, 2021

kibanamachine removed v8.1.0 skipped-test blocker labels Dec 14, 2021

kibanamachine reopened this Aug 17, 2022

spalger mentioned this issue Aug 25, 2022

[jest/helpers] add safeAct helper to try and debug act() #139444

Closed

jbudz added a commit that referenced this issue Sep 20, 2022

skip flaky suite. #115307

e468632

jbudz added blocker skipped-test v8.5.0 labels Sep 20, 2022

Dosant mentioned this issue Sep 27, 2022

Run ILM jest tests as integration tests allowing them to run beyond 5s timeout #141750

Merged

Dosant closed this as completed in #141750 Sep 28, 2022

kibanamachine removed skipped-test v8.5.0 blocker labels Sep 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing test: Jest Tests.x-pack/plugins/index_lifecycle_management/jest/client_integration/edit_policy/form_validation - <EditPolicy /> timing validation should validate that min_age is equal or greater than previous phase min_age #115307

Failing test: Jest Tests.x-pack/plugins/index_lifecycle_management/jest/client_integration/edit_policy/form_validation - <EditPolicy /> timing validation should validate that min_age is equal or greater than previous phase min_age #115307

kibanamachine commented Oct 17, 2021 •

edited

Loading

elasticmachine commented Oct 18, 2021

kibanamachine commented Oct 28, 2021

spalger commented Oct 28, 2021

kibanamachine commented Oct 30, 2021

kibanamachine commented Nov 12, 2021

kibanamachine commented Aug 17, 2022

spalger commented Aug 17, 2022

kibanamachine commented Aug 23, 2022

sebelga commented Aug 24, 2022

sebelga commented Aug 24, 2022 •

edited

Loading

spalger commented Aug 24, 2022

spalger commented Aug 25, 2022

sebelga commented Aug 26, 2022

spalger commented Aug 30, 2022 •

edited

Loading

spalger commented Aug 31, 2022

spalger commented Aug 31, 2022

spalger commented Sep 1, 2022

sebelga commented Sep 5, 2022

spalger commented Sep 6, 2022

sebelga commented Sep 8, 2022

kibanamachine commented Sep 20, 2022

jbudz commented Sep 20, 2022

Failing test: Jest Tests.x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/form_validation - <EditPolicy /> timing validation should validate that min_age is equal or greater than previous phase min_age #115307

Failing test: Jest Tests.x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/form_validation - <EditPolicy /> timing validation should validate that min_age is equal or greater than previous phase min_age #115307

Comments

kibanamachine commented Oct 17, 2021 • edited Loading

elasticmachine commented Oct 18, 2021

kibanamachine commented Oct 28, 2021

spalger commented Oct 28, 2021

kibanamachine commented Oct 30, 2021

kibanamachine commented Nov 12, 2021

kibanamachine commented Aug 17, 2022

spalger commented Aug 17, 2022

kibanamachine commented Aug 23, 2022

sebelga commented Aug 24, 2022

sebelga commented Aug 24, 2022 • edited Loading

spalger commented Aug 24, 2022

spalger commented Aug 25, 2022

sebelga commented Aug 26, 2022

spalger commented Aug 30, 2022 • edited Loading

spalger commented Aug 31, 2022

spalger commented Aug 31, 2022

spalger commented Sep 1, 2022

sebelga commented Sep 5, 2022

spalger commented Sep 6, 2022

sebelga commented Sep 8, 2022

kibanamachine commented Sep 20, 2022

jbudz commented Sep 20, 2022

Failing test: Jest Tests.x-pack/plugins/index_lifecycle_management/jest/client_integration/edit_policy/form_validation - <EditPolicy /> timing validation should validate that min_age is equal or greater than previous phase min_age #115307

Failing test: Jest Tests.x-pack/plugins/index_lifecycle_management/jest/client_integration/edit_policy/form_validation - <EditPolicy /> timing validation should validate that min_age is equal or greater than previous phase min_age #115307

kibanamachine commented Oct 17, 2021 •

edited

Loading

sebelga commented Aug 24, 2022 •

edited

Loading

spalger commented Aug 30, 2022 •

edited

Loading