Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test code - dashboard add panel and accident clicks on add new visualization #20819

Closed
stacey-gammon opened this issue Jul 16, 2018 · 7 comments · Fixed by #21109
Closed

Flaky test code - dashboard add panel and accident clicks on add new visualization #20819

stacey-gammon opened this issue Jul 16, 2018 · 7 comments · Fixed by #21109
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Feature:Dashboard Dashboard related features Team:Visualizations Visualization editors, elastic-charts and infrastructure test

Comments

@stacey-gammon
Copy link
Contributor

@bmcconaghy ran into a flaky test locally and we figured out that what was happening (or at least strongly suspect) is that the flyout transitioning from left to right and simulatenously having selenium attempt to type in a visualization to search for was causing accidental navigation to the add visualization page.

The screenshot is one of the visualization page, hence it can't find the search input box anymore. We should modify the tests so that when we open or close a flyout we wait for the transition to be complete before continuing on with any other test code.

Doesn't seem to be causing failures on master (yet), but I wouldn't be surprised if it starts at some point.

cc @liza-mae (since you've been working on a lot of flaky test stuff, thought you might want to be aware of this one).

Output:

   │ info  [logstash_functional] Loading "data.json.gz"

       │ info  [logstash_functional] Skipped restore for existing index "logstash-2015.09.22"

       │ info  [logstash_functional] Skipped restore for existing index "logstash-2015.09.20"

       │ info  [logstash_functional] Skipped restore for existing index "logstash-2015.09.21"

       │ info  [logstash_functional] Indexed 0 docs into "logstash-2015.09.22"

       │ info  [logstash_functional] Indexed 0 docs into "logstash-2015.09.20"

       │ info  [logstash_functional] Indexed 0 docs into "logstash-2015.09.21"

       │ info  [dashboard_view_mode] Loading "mappings.json"

       │ info  [dashboard_view_mode] Loading "data.json.gz"

       │ info  [dashboard_view_mode] Deleted existing index ".kibana"

       │ info  [dashboard_view_mode] Created index ".kibana"

       │ debg  [dashboard_view_mode] ".kibana" settings {"index":{"number_of_shards":"1","number_of_replicas":"1"}}

       │ info  [dashboard_view_mode] Indexed 8 docs into ".kibana"

       │ debg  replacing kibana config doc: {"dateFormat:tz":"UTC","defaultIndex":"logstash-*"}

       │ debg  applying update to kibana config: {"notifications:lifetime:banner":3600000,"notifications:lifetime:error":3600000,"notifications:lifetime:warning":3600000,"notifications:lifetime:info":3600000}

       │ debg  navigating to dashboard url: http://localhost:5640/app/kibana#/dashboards

       │ debg  navigate to: http://localhost:5640/app/kibana#/dashboards

       │ debg  ... sleep(700) start

       │ debg  ... sleep(700) end

       │ debg  returned from get, calling refresh

       │ debg  currentUrl = http://localhost:5640/login?next=%2Fapp%2Fkibana%3F_t%3D1531745414681#/dashboards

       │           appUrl = http://localhost:5640/app/kibana#/dashboards

       │ debg  Found login page.  Logging in with username = elastic

       │ debg  Finished login process currentUrl = http://localhost:5640/app/kibana#/dashboards?_g=()

       │ debg  TestSubjects.find(kibanaChrome)

       │ debg  findByCssSelector [data-test-subj~="kibanaChrome"]

       │ debg  ... sleep(501) start

       │ debg  ... sleep(501) end

       │ debg  in navigateTo url = http://localhost:5640/app/kibana#/dashboards?_g=()

       │ debg  TestSubjects.exists(statusPageContainer)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="statusPageContainer"]

       │ debg  TestSubjects.exists(newDashboardLink)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="newDashboardLink"]

       │ debg  TestSubjects.click(createDashboardPromptButton)

       │ debg  TestSubjects.find(createDashboardPromptButton)

       │ debg  findByCssSelector [data-test-subj~="createDashboardPromptButton"]

       │ debg  DashboardAddPanel.addVisualizations

       │ debg  DashboardAddPanel.addVisualization(Visualization PieChart)

       │ debg  DashboardAddPanel.ensureAddPanelIsShowing

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.clickOpenAddPanel

       │ debg  TestSubjects.click(dashboardAddPanelButton)

       │ debg  TestSubjects.find(dashboardAddPanelButton)

       │ debg  findByCssSelector [data-test-subj~="dashboardAddPanelButton"]

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  TestSubjects.find(savedObjectFinderSearchInput)

       │ debg  findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]

       │ debg  isGlobalLoadingIndicatorVisible

       │ debg  TestSubjects.exists(globalLoadingIndicator)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="globalLoadingIndicator"]

       │ debg  awaitGlobalLoadingIndicatorHidden

       │ debg  TestSubjects.find(globalLoadingIndicator-hidden)

       │ debg  findByCssSelector [data-test-subj~="globalLoadingIndicator-hidden"]

       │ debg  TestSubjects.click(addPanelVisualization-PieChart)

       │ debg  TestSubjects.find(addPanelVisualization-PieChart)

       │ debg  findByCssSelector [data-test-subj~="addPanelVisualization-PieChart"]

       │ debg  DashboardAddPanel.closeAddPanel

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  TestSubjects.find(dashboardAddPanel)

       │ debg  findByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.addVisualization(Visualization☺ VerticalBarChart)

       │ debg  DashboardAddPanel.ensureAddPanelIsShowing

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.clickOpenAddPanel

       │ debg  TestSubjects.click(dashboardAddPanelButton)

       │ debg  TestSubjects.find(dashboardAddPanelButton)

       │ debg  findByCssSelector [data-test-subj~="dashboardAddPanelButton"]

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  TestSubjects.find(savedObjectFinderSearchInput)

       │ debg  findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]

       │ debg  isGlobalLoadingIndicatorVisible

       │ debg  TestSubjects.exists(globalLoadingIndicator)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="globalLoadingIndicator"]

       │ debg  awaitGlobalLoadingIndicatorHidden

       │ debg  TestSubjects.find(globalLoadingIndicator-hidden)

       │ debg  findByCssSelector [data-test-subj~="globalLoadingIndicator-hidden"]

       │ debg  TestSubjects.click(addPanelVisualization☺-VerticalBarChart)

       │ debg  TestSubjects.find(addPanelVisualization☺-VerticalBarChart)

       │ debg  findByCssSelector [data-test-subj~="addPanelVisualization☺-VerticalBarChart"]

       │ debg  DashboardAddPanel.closeAddPanel

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  TestSubjects.find(dashboardAddPanel)

       │ debg  findByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.addVisualization(Visualization漢字 AreaChart)

       │ debg  DashboardAddPanel.ensureAddPanelIsShowing

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.clickOpenAddPanel

       │ debg  TestSubjects.click(dashboardAddPanelButton)

       │ debg  TestSubjects.find(dashboardAddPanelButton)

       │ debg  findByCssSelector [data-test-subj~="dashboardAddPanelButton"]

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  TestSubjects.find(savedObjectFinderSearchInput)

       │ debg  findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]

       │ debg  isGlobalLoadingIndicatorVisible

       │ debg  TestSubjects.exists(globalLoadingIndicator)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="globalLoadingIndicator"]

       │ debg  awaitGlobalLoadingIndicatorHidden

       │ debg  TestSubjects.find(globalLoadingIndicator-hidden)

       │ debg  findByCssSelector [data-test-subj~="globalLoadingIndicator-hidden"]

       │ debg  TestSubjects.click(addPanelVisualization漢字-AreaChart)

       │ debg  TestSubjects.find(addPanelVisualization漢字-AreaChart)

       │ debg  findByCssSelector [data-test-subj~="addPanelVisualization漢字-AreaChart"]

       │ debg  DashboardAddPanel.closeAddPanel

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  TestSubjects.find(dashboardAddPanel)

       │ debg  findByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.addVisualization(Visualization☺漢字 DataTable)

       │ debg  DashboardAddPanel.ensureAddPanelIsShowing

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  DashboardAddPanel.clickOpenAddPanel

       │ debg  TestSubjects.click(dashboardAddPanelButton)

       │ debg  TestSubjects.find(dashboardAddPanelButton)

       │ debg  findByCssSelector [data-test-subj~="dashboardAddPanelButton"]

       │ debg  DashboardAddPanel.isAddPanelOpen

       │ debg  TestSubjects.exists(dashboardAddPanel)

       │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]

       │ debg  TestSubjects.find(savedObjectFinderSearchInput)

       │ debg  findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]

       │ debg  --- tryForTime error: [POST http://localhost:9515/session/9d77867f16dafd70b47c64fec17285dd/element/0.46767505709334767-20/clear] invalid element state: Element must be user-editable in order to clear it.

       │         (Session info: chrome=67.0.3396.99)

       │         (Driver info: chromedriver=2.36.540469 (1881fd7f8641508feb5166b7cae561d87723cfa8),platform=Mac OS X 10.13.4 x86_64)

       │ debg  TestSubjects.find(savedObjectFinderSearchInput)

       │ debg  findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]

       │ debg  --- tryForTime error: [POST http://localhost:9515/session/9d77867f16dafd70b47c64fec17285dd/element / {"using":"css selector","value":"[data-test-subj~=\"savedObjectFinderSearchInput\"]"}] no such element: Unable to locate element: {"method":"css selector","selector":"[data-test-subj~="savedObjectFinderSearchInput"]"}

       │         (Session info: chrome=67.0.3396.99)

       │         (Driver info: chromedriver=2.36.540469 (1881fd7f8641508feb5166b7cae561d87723cfa8),platform=Mac OS X 10.13.4 x86_64)

       │ debg  --- tryForTime errored again with the same message  ...

       │ debg  --- tryForTime errored again with the same message  ...

       │ debg  --- tryForTime errored again with the same message  ...
....
       │ debg  --- tryForTime errored again with the same message  ...

       │ debg  --- tryForTime error: tryForTime timeout: [POST http://localhost:9515/session/9d77867f16dafd70b47c64fec17285dd/element / {"using":"css selector","value":"[data-test-subj~=\"savedObjectFinderSearchInput\"]"}] no such element: Unable to locate element: {"method":"css selector","selector":"[data-test-subj~="savedObjectFinderSearchInput"]"}

       │         (Session info: chrome=67.0.3396.99)

       │         (Driver info: chromedriver=2.36.540469 (1881fd7f8641508feb5166b7cae561d87723cfa8),platform=Mac OS X 10.13.4 x86_64)

       │           at Server._post (/Users/bmcconaghy/workspace/kibana/test/functional/services/remote/verbose_remote_logging.js:34:21)

       │           at runRequest (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:92:40)

       │           at /Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:113:39

       │           at new Promise (/Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/Promise.ts:411:4)

       │           at Session._post (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:67:10)

       │           at Session.find (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:1328:15)

       │           at Command.<anonymous> (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Command.js:42:36)

       │           at /Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/Promise.ts:393:16

       │           at run (/Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/Promise.ts:237:8)

       │           at /Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/nextTick.ts:44:4

       │           at _combinedTickCallback (internal/process/next_tick.js:131:7)

       │           at process._tickCallback (internal/process/next_tick.js:180:9)

       │           at Command.find (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Command.js:23:10)

       │           at Command.prototype.(anonymous function) [as findByCssSelector] (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/lib/strategies.js:29:16)

       │           at _ensureElementWithTimeout (/Users/bmcconaghy/workspace/kibana/test/functional/services/find.js:71:29)

       │           at retry.try (/Users/bmcconaghy/workspace/kibana/test/functional/services/find.js:51:33)

       │           at tryCatcher (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/util.js:26:23)

       │           at Function.Promise.attempt.Promise.try (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/method.js:31:24)

       │           at attempt (/Users/bmcconaghy/workspace/kibana/test/common/services/retry.js:41:16)

       │           at tryCatcher (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/util.js:26:23)

       │           at Promise._settlePromiseFromHandler (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:503:31)

       │           at Promise._settlePromiseAt (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:577:18)

       │           at Promise._settlePromises (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:693:14)

       │           at Async._drainQueue (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:123:16)

       │           at Async._drainQueues (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:133:10)

       │           at Immediate.Async.drainQueues [as _onImmediate] (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:15:14)

       │           at runCallback (timers.js:810:20)

       │           at tryOnImmediate (timers.js:768:5)

       │           at processImmediate [as _immediateCallback] (timers.js:745:5)

       │ info  Taking screenshot "/Users/bmcconaghy/workspace/kibana/x-pack/test/functional/screenshots/failure/dashboard mode Dashboard View Mode _before all_ hook_ initialize tests.png"

       │ info  Current URL is: http://localhost:5640/app/kibana#/visualize/new?_g=()

       │ info  Saving page source to: /Users/bmcconaghy/workspace/kibana/x-pack/test/functional/failure_debug/html/dashboard mode Dashboard View Mode _before all_ hook_ initialize tests.html

     └- ✖ fail: "dashboard mode Dashboard View Mode "before all" hook: initialize tests"

     │        tryForTime timeout: Error: tryForTime timeout: [POST http://localhost:9515/session/9d77867f16dafd70b47c64fec17285dd/element / {"using":"css selector","value":"[data-test-subj~=\"savedObjectFinderSearchInput\"]"}] no such element: Unable to locate element: {"method":"css selector","selector":"[data-test-subj~="savedObjectFinderSearchInput"]"}

     │         (Session info: chrome=67.0.3396.99)

     │         (Driver info: chromedriver=2.36.540469 (1881fd7f8641508feb5166b7cae561d87723cfa8),platform=Mac OS X 10.13.4 x86_64)

     │           at Server._post (/Users/bmcconaghy/workspace/kibana/test/functional/services/remote/verbose_remote_logging.js:34:21)

     │           at runRequest (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:92:40)

     │           at /Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:113:39

     │           at new Promise (/Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/Promise.ts:411:4)

     │           at Session._post (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:67:10)

     │           at Session.find (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:1328:15)

     │           at Command.<anonymous> (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Command.js:42:36)

     │           at /Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/Promise.ts:393:16

     │           at run (/Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/Promise.ts:237:8)

     │           at /Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/nextTick.ts:44:4

     │           at _combinedTickCallback (internal/process/next_tick.js:131:7)

     │           at process._tickCallback (internal/process/next_tick.js:180:9)

     │           at Command.find (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Command.js:23:10)

     │           at Command.prototype.(anonymous function) [as findByCssSelector] (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/lib/strategies.js:29:16)

     │           at _ensureElementWithTimeout (/Users/bmcconaghy/workspace/kibana/test/functional/services/find.js:71:29)

     │           at retry.try (/Users/bmcconaghy/workspace/kibana/test/functional/services/find.js:51:33)

     │           at tryCatcher (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/util.js:26:23)

     │           at Function.Promise.attempt.Promise.try (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/method.js:31:24)

     │           at attempt (/Users/bmcconaghy/workspace/kibana/test/common/services/retry.js:41:16)

     │           at tryCatcher (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/util.js:26:23)

     │           at Promise._settlePromiseFromHandler (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:503:31)

     │           at Promise._settlePromiseAt (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:577:18)

     │           at Promise._settlePromises (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:693:14)

     │           at Async._drainQueue (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:123:16)

     │           at Async._drainQueues (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:133:10)

     │           at Immediate.Async.drainQueues [as _onImmediate] (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:15:14)

     │           at runCallback (timers.js:810:20)

     │           at tryOnImmediate (timers.js:768:5)

     │           at processImmediate [as _immediateCallback] (timers.js:745:5)

     │           at attempt (/Users/bmcconaghy/workspace/kibana/test/common/services/retry.js:38:17)

     │           at tryCatcher (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/util.js:26:23)

     │           at Promise._settlePromiseFromHandler (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:503:31)

     │           at Promise._settlePromiseAt (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:577:18)

     │           at Promise._settlePromises (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/promise.js:693:14)

     │           at Async._drainQueue (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:123:16)

     │           at Async._drainQueues (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:133:10)

     │           at Immediate.Async.drainQueues (/Users/bmcconaghy/workspace/kibana/node_modules/bluebird/js/main/async.js:15:14)

     │           at runCallback (timers.js:810:20)

     │           at tryOnImmediate (timers.js:768:5)

     │           at processImmediate [as _immediateCallback] (timers.js:745:5)

     │         Error: tryForTime timeout: Error: tryForTime timeout: [POST http://localhost:9515/session/9d77867f16dafd70b47c64fec17285dd/element / {"using":"css selector","value":"[data-test-subj~=\"savedObjectFinderSearchInput\"]"}] no such element: Unable to locate element: {"method":"css selector","selector":"[data-test-subj~="savedObjectFinderSearchInput"]"}

     │           (Session info: chrome=67.0.3396.99)

     │           (Driver info: chromedriver=2.36.540469 (1881fd7f8641508feb5166b7cae561d87723cfa8),platform=Mac OS X 10.13.4 x86_64)

     │         at Server._post (/Users/bmcconaghy/workspace/kibana/test/functional/services/remote/verbose_remote_logging.js:34:21)

     │         at runRequest (/Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:92:40)

     │         at /Users/bmcconaghy/workspace/kibana/node_modules/leadfoot/Session.js:113:39

     │         at new Promise (/Users/bmcconaghy/workspace/kibana/node_modules/dojo/_debug/Promise.ts:411:4)
...
@stacey-gammon stacey-gammon added test Feature:Dashboard Dashboard related features :Sharing labels Jul 16, 2018
@LeeDr
Copy link

LeeDr commented Jul 23, 2018 •
edited
Loading

I'm seeing similar occasional failures running tests locally from master.

My thought is that when the test clicks the Add menu item and that panel flys out, there's a query to Elasticsearch for the list of visualizations and the savedObjectFinderSearchInput isn't editable until that query returns.
But the try loop is higher up in the stack and takes some other action when this failure happens, which leads to getting to the Visualization / New page.

         │ debg  TestSubjects.find(savedObjectFinderSearchInput)
         │ debg  findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]
         │ debg  --- tryForTime error: [POST http://localhost:9515/session/e4a7ea85e1cee84b0c7437919535cfb6/element/0.26627894985528733-19/clear] invalid element state: Element must be user-editable in order to clear it.
         │         (Session info: chrome=67.0.3396.99)
         │         (Driver info: chromedriver=2.36.540471 (9c759b81a907e70363c6312294d30b6ccccc2752),platform=Linux 4.15.0-20-generic x86_64)
         │ debg  TestSubjects.find(savedObjectFinderSearchInput)
         │ debg  findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]
         │ debg  --- tryForTime error: [POST http://localhost:9515/session/e4a7ea85e1cee84b0c7437919535cfb6/element / {"using":"css selector","value":"[data-test-subj~=\"savedObjectFinderSearchInput\"]"}] no such element: Unable to locate element: {"method":"css selector","selector":"[data-test-subj~="savedObjectFinderSearchInput"]"}
         │         (Session info: chrome=67.0.3396.99)
         │         (Driver info: chromedriver=2.36.540471 (9c759b81a907e70363c6312294d30b6ccccc2752),platform=Linux 4.15.0-20-generic x86_64)
         │ debg  --- tryForTime errored again with the same message  ...
         │ debg  --- tryForTime errored again with the same message  ...

I'm trying a try loop in services dashboard add_panel filterEmbeddableNames(name)

@liza-mae liza-mae mentioned this issue Jul 23, 2018
Closed
@LeeDr
Copy link

LeeDr commented Jul 23, 2018

Discovered that there is a loading indicator on the table. While it's loading it has these classes;
euiBasicTable euiBasicTable-loading and when loading is done it's just euiBasicTable.

So we should add code to wait for that loading indicator to go away before trying to setValue on the filter.

I'll work on a PR for it.

@LeeDr LeeDr mentioned this issue Jul 23, 2018
Merged
@stacey-gammon
Copy link
Contributor Author

@LeeDr Mind if I re-open this? I suspect that PR, while helpful, did not fix the issue with flyouts.

@stacey-gammon stacey-gammon reopened this Jul 26, 2018
@LeeDr
Copy link

LeeDr commented Jul 26, 2018

@stacey-gammon did you actually figure out how this could happen?
" flyout transitioning from left to right and simulatenously having selenium attempt to type in a visualization to search for was causing accidental navigation to the add visualization page."

@stacey-gammon
Copy link
Contributor Author

nope, just a suspicion. I'm attempting to isolate some of these dashboard tests to see if the failure still exists after your changes to verify, but being pulled into test reviews, so haven't had time yet. I have this PR which passed the last run: #21225

But perhaps I should try running one that was being hit more often, or the Dashboard View Mode suite.

@timroes timroes added Team:Visualizations Visualization editors, elastic-charts and infrastructure and removed :Sharing labels Sep 13, 2018
@rayafratkina
Copy link
Contributor

Removing the version labels: we should not be assigning version until the PR is merged.

@liza-mae liza-mae added failed-test A test failure on a tracked branch, potentially flaky-test and removed flaky-failing-test labels Nov 1, 2018
@stacey-gammon
Copy link
Contributor Author

Pretty sure this has been fixed when animations was disabled

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Feature:Dashboard Dashboard related features Team:Visualizations Visualization editors, elastic-charts and infrastructure test
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add method to wait for loading to complete in Add Panel table LeeDr/kibana
6 participants
@snide @timroes @LeeDr @stacey-gammon @liza-mae @rayafratkina