-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add method to wait for loading to complete in Add Panel table #21109
Conversation
💔 Build Failed |
There was a failure in an ML x-pack test. I pinged the ML team but it's late in the day for them now. I'm going to have jenkins test this again just in case it's intermittent or something. Otherwise we might have to skip or ignore the failing test. |
💔 Build Failed |
await this.waitForEuiTableLoading(); | ||
// This try loop lets us recover it the field wasn't enabled yet | ||
await retry.try(async () => { | ||
await testSubjects.setValue('savedObjectFinderSearchInput', name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this extra retry loop adds anything because there is already a retry loop on the inside of setValue:
async setValue(selector, text) {
return await retry.try(async () => {
await this.click(selector);
// in case the input element is actually a child of the testSubject, we
// call clearValue() and type() on the element that is focused after
// clicking on the testSubject
const input = await remote.getActiveElement();
await input.clearValue();
await input.type(text);
});
}
We could think about confirming that the value was actually set by adding something in that function call that does like:
async setValue(selector, text) {
return await retry.try(async () => {
await this.click(selector);
// in case the input element is actually a child of the testSubject, we
// call clearValue() and type() on the element that is focused after
// clicking on the testSubject
const input = await remote.getActiveElement();
await input.clearValue();
await input.type(text);
const newValue = await this.getVisibleText(selector);
if (newValue != text) {
throw new Error('Value was not set');
}
});
}
But I'm not sure if getVisibleText
is what we'd want or if it's work on all element types (might be something like getValue we should use?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't realize that setValue method had a retry loop. But that makes me wonder why it wasn't solving this problem from the beginning. The first error in this case was coming from the clearValue() step in that setValue method;
invalid element state: Element must be user-editable in order to clear it.
But then the next message says that it failed to find the element.
no such element: Unable to locate element: {"method":"css selector","selector":"[data-test-subj~="savedObjectFinderSearchInput"]"}
And when the test finally fails and takes a screenshot, we see that it has navigated away from the Add Panel flyout and has opened the New visualization page (which might explain why it can no longer find the element). It seems as if this inner-most retry isn't staying in this loop for some reason, and an outer retry is trying to clicking on "Add" again, or somewhere else that causes the incorrect navigation.
What you don't see here is the debug logging I had in place which did show that the euiBasicTable did initially show the euiBasicTable-loading
class every time we opened the Add Panel flyout and that it was gone after we waited for it to be deleted;
│ debg DashboardAddPanel.isAddPanelOpen
│ debg TestSubjects.exists(dashboardAddPanel)
│ debg existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]
│ debg TestSubjects.find(dashboardAddPanel)
│ debg findByCssSelector [data-test-subj~="dashboardAddPanel"]
│ debg ################################################################# euiBasicTable euiBasicTable-loading
│ debg ##############################++++++++++++++++++++++++++++++++++++++ euiBasicTable
│ debg TestSubjects.find(savedObjectFinderSearchInput)
│ debg findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]
│ debg isGlobalLoadingIndicatorVisible
│ debg TestSubjects.exists(globalLoadingIndicator)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, so my suspicion is that the mechanism for typing selenium uses is accidentally hitting the Add Visualization
button. Once the page navigates away, doesn't matter how many times it loops, it's going to fail. That's why I think we have to add a function that waits till the flyout finishes it's transition before we even start typing into that field.
@@ -93,6 +93,11 @@ export function DashboardAddPanelProvider({ getService, getPageObjects }) { | |||
} | |||
} | |||
|
|||
async waitForEuiTableLoading() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like this function could go somewhere more general as it might be useful for other places that use the eui table and want to wait for it to load. I don't see a good spot for it right now. Maybe a new service named euiComponents
? Could take in an optional selector name to use to narrow down but I wonder if you can use it without an element, directly on remote (just like we do for the global loading indicator - making sure it doesn't exist at all, anywhere on the page).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about that. But the euiBasicTable can get populated from any data source and doesn't necessarily require a request/response to Elasticsearch so I'm not sure what to do.
Again, the retry loop in setValue seems like it should always handle the case where the field isn't initially user-editable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like this function could go somewhere more general as it might be useful for other places that use the eui table and want to wait for it to load. I don't see a good spot for it right now. Maybe a new service named euiComponents?
There is a flyout service and I have a PR out for a combo box service. #21219.
I like the addition of the waitForEuiTableLoading, but I don't know if that will solve this particular issue (or all of them anyway - might solve one). I think we need to find a way to ensure that the flyout has finished it's transition from left to right. Could maybe add a function to |
retest |
@stacey-gammon I'm not sure the flying-out action has any impact on this problem. It's not like we ever see a failure screenshot with the fly-out half way out? In my test runs, I haven't seen any retry loops get hit in this area of code. Not the inner-most one in setValue, not the new one I added in filterEmbeddableNames, and not any outer ones (which I think cause the incorrect navigation). So that makes me think that waiting for the table loading to complete solves the flakiness of some of these tests. |
Well that's cause the visualization page would finish loading during the retry. I have seen the visualization page screenshot which never made sense if it was just adding panels to a dashboard.
I'm happy to check this in to solve the flakiness of some of these tests at least, and the table load wait makes sense to me. I just still suspect another issue with the flyout transition. But we can check this in first and see what happens with the other flaky issues. |
@stacey-gammon I had a thought. If I temporarily tweaked the fly-out time so that it went really slow, like 2 or 3 seconds to fly-out so that the table loading had already finished before the fly-out was even half way out. Then we could see if that matters or not. I'm guessing it won't matter, but I've been wrong before. |
@LeeDr, what is your goal by doing that? are you trying to hit the bug or see if it solves it? If the latter, you could try to completely remove the transition and see if that fixes it. If you mean the former... you could try it but I don't think it'd offer an obvious answer since it's flaky. Well, neither would give obvious answers since it's flaky. I think perhaps step one would be to see if a single test failure can be consistently reproduced by modifying the code so it jenkins runs a single test suite a bunch of times. Historically I've done this like this: c09460f Though given how many other flaky tests there are, may also need to pair that with commenting out some of the other tests to focus only on fixing a single issue at a time. |
💔 Build Failed |
💚 Build Succeeded |
💚 Build Succeeded |
…c#21109) * Add method to wait for loading to complete in Add Panel table * remove extra retry loop
…c#21109) * Add method to wait for loading to complete in Add Panel table * remove extra retry loop
#21246) * Add method to wait for loading to complete in Add Panel table * remove extra retry loop
#21247) * Add method to wait for loading to complete in Add Panel table * remove extra retry loop
btw @LeeDr - haven't had time to investigate more but I noticed while running the tests locally that there is a significant wait (couple seconds) right before typing into the add panel. I'm thinking it might be from this PR. I wonder if that is why it appeared to fix the open flyout issue. Maybe it didn't fix the root cause but just fixed it because it was equivalent to adding a long sleep. Anyway, as long as tests are not flaky, that's an improvement, but might be worth circling back around to this at some point in the future. |
Fixes #20819
The new euiBasicTable is used in the Dashboard Add Panel flyout. When it opens it queries for the list of visualizations to show in the table. It shows a moving blue progress bar during that time. If the test tries to filter the visualizations by entering text into the savedObjectFinderSearchInput before it's ready it can fail (see issue above).
This PR changes 3 things;
adds a new method
waitForEuiTableLoading
which waits doeswaitForDeletedByClassName('euiBasicTable-loading')
calls that wait method in
filterEmbeddableNames(name)
puts the
testSubjects.setValue('savedObjectFinderSearchInput', name);
inside a try loop. This change by itself should probably be enough to fix the issue, but I'd rather wait for the loading indicator than see tryForTime failures in the log.